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CHAPTER  1 


INTRODUCTION 

Overview 

In  March  of  1977,  the  development  of  the  Merit  Promotion 
Appraisal  System  (MPAS)  was  begun  by  and  for  the  Air  Force  Systems 
Command  (AFSC)  at  Wright-Patterson  AFB,  Ohio.  The  Commander  of  the 
Aeronautical  Systems  Division  (ASD)  is  the  ranking  AFSC  Commander  on 
the  base  and  has  certain  responsibilities  for  all  AFSC  activities. 
Among  these  responsibilities  is  personnel  management  of  civilian  and 
military  employees  (15:3). 

When  civilian  vacancies  occur,  all  qualified  AFSC  civilian 
employees  at  Wricht-Patterson  AFB  are  eligible  for  consideration 
through  merit  promotion.  The  selecting  supervisor  is  provided  a  list 
of  the  top  three  to  five  eligible  candidates  for  each  vacancy.  Candi¬ 
dates  are  ranked  by  merit  appraisals,  training,  education,  and  experi¬ 
ence.  Since  the  merit  appraisal  comprises  approximately  half  of  the 
total  points,  the  validity  and  acceptability  of  the  ratings  across 
AFSC  activities  is  of  major  concern  to  both  the  employees  and  AFSC 
managers  (15:4). 

Prior  to  the  MPAS,  merit  appraisals  had  become  inflated  and 
showed  little  variance  from  one  employee  to  another.  It  had  become 
impossible  to  distinguish  an  exceptionally  productive  employee  from 
the  fair  or  low-oroducing  employee.  The  MPAS  had  the  objective  of 
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obtaining  a  more  valid  measure  of  employees'  abilities  and  providing 
feedback  (15:4). 

Under  MPAS,  enployees  are  rated  through  use  of  Behaviorly 
Based  Rating  Scales  (BBRS).  For  technical  and  professional  enployees, 
MPAS  measures  eleven  factors  as  perceived  by  the  immediate  supervisor 
and  reviewed  by  the  next  higher  level  supervisor.  Professional 
enployees  are  those  enployees  that  not  only  have  the  knowledge  to  do 
technical  jobs  but  also  are  in  positions  which  require  seme  management 
expertise.  These  enployees  usually  are  in  a  double  interval  promotion 
system.  For  example,  they  may  start  as  a  GS-5  with  promotion  steps 
to  GS-7,  GS-9,  and  GS-11.  Technical  enployees  are  only  required  to 
have  the  technical  expertise  for  their  jobs  and  are  in  a  single  inter¬ 
val  oromotion  system.  They  usually  have  pay  grades  from  GS-5  through 
GS-9  and  are  promoted  only  one  pay  grade  at  a  time.  Clerical  per¬ 
sonnel  are  similarly  rated  on  eight  different  factors.  Exanples  of 
the  rating  forms  are  shown  in  Appendices  A,  B,  and  C.  To  obtain  an 
overall  merit  appraisal  rating,  the  factors  are  averaged  and  multi¬ 
plied  by  ten.  Since  the  BBRS  are  on  a  scale  of  one  to  seven  points 
for  each  factor,  the  maximum  possible  overall  merit  appraisal  rating 
is  seventy.  This  overall  merit  appraisal  rating  is  one  input  for 
ranking  eligible  individuals  for  any  given  job. 

After  the  MPAS  was  fully  developed,  it  was  implemented  in  the 
EN  division  of  ASD  in  1977.  In  early  1978,  the  Air  Force  Wright 
Aeronautical  Laboratory  (AFWAL)  also  began  using  MPAS  for  employee 
ratings.  Finally,  in  late  1978,  all  civilian  personnel  in  ASD  and 
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AFWAL  were  rated  under  the  MPAS.  This  progressive  implementation  of 
the  MPAS  was  likewise  accompanied  by  training  of  affected  personnel 
to  increase  acceptance  of  the  rating  system.  The  training  provided 
management  personnel  experience  in  using  the  system  and  increased 
their  appraisal  skills. 

The  cornnon  tendency  of  supervisors  to  rate  all  subordinates 
the  same,  or  nearly  the  same,  makes  it  difficult  to  recognize  indi¬ 
viduals  deserving  promotion.  If  the  obtained  ratings  are  not  useful, 
other  secondary  factors  are  used  to  indicate  promo tability;  i.e., 
training,  experience,  seniority.  Therefore,  it  is  essential  that  any 
rating  system  used  for  promotion  of  individuals  be  as  valid  and  dis¬ 
cerning  as  possible  (2:6). 

To  obtain  information  useful  in  the  development  of  any  new 
appraisal  system  that  may  be  forthcoming,  an  analysis  of  the  strengths 
and  weaknesses  of  the  MPAS  should  be  helpful.  A  large  data  base  of 
information  is  available  for  the  effective  evaluation  of  the  MPAS. 

The  ratings  obtained  in  the  appraisals  from  1977  through  1980  contain 
not  only  each  individual  factor  used  in  the  rating,  but  also  the 
individual's  age,  sex,  organization,  grade,  and  race.  Information 
obtained  from  follow-up  questionnaires  in  1977,  1978,  and  1979  is 
available  for  use  in  determining  how  well  the  MPAS  was  accepted  by  the 
enployees.  These  questionnaires  were  administered  to  the  managers 
rating  enployees  on  the  MPAS  and  to  enployees  rated  by  the  MPAS  to 
provide  feedback  on  the  system.  Finally,  data  obtained  for  a  study 
to  conpare  self  ratings,  peer  ratings,  supervisory  ratings,  and 
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ratings  by  a  manager  who  is  not  the  employee  *s  direct  supervisor  is 
available  that  would  be  very  useful  in  examining  the  validity  of  MPAS. 


Objective 

Since  the  objective  of  the  MPAS  was  to  provide  a  promotion 
rating  system  that  would  discern  between  individuals  deserving  promo¬ 
tion  and  those  individuals  that  do  not,  the  objective  of  this  study 
is  to  determine  some  of  the  strengths  and  weaknesses  of  the  MPAS 
through  analysis  of  the  available  data. 

These  sub-objectives  are  used  in  the  analysis:  (1)  trend 
analysis  of  the  promo tability  ratings  from  1975  through  1980,  (2) 
validity  analysis  of  the  overall  promo  tability  rating  and  rating 
factors  of  the  data  from  the  supervisor  ratings,  self-ratings,  peer 
ratings,  and  alternate-supervisor  ratings,  and  (3)  acceptance  of  the 
MPAS  as  a  valid  measurement  of  promo  tability. 

Summary 

This  chapter  has  presented  a  brief  description  of  the  MPAS 
and  its  development.  Also  included  in  this  chapter  are  the  objectives 
which  this  thesis  pursues.  In  Chapter  2,  more  background  into  the 
development  of  the  MPAS  is  given.  The  methodology  used  in  the  accom¬ 
plishment  of  the  stated  objectives  is  presented  in  Chapter  3, 

Chapter  A  contains  the  results  of  the  data  analysis  using  the 
presented  methodology.  Finally,  Chapter  5  presents  the  conclusions 
and  recommendations  derived  from  the  data  analysis. 


CHAPTER  2 


BACKGROUND 

Overview 

A  good  manager  is  naturally  concerned  about  the  overall 
performance  of  subordinates.  In  order  to  assess  the  promotability  of 
a  job  candidate  into  a  specific  job,  it  is  inportant  to  assess  the 
candidate's  specific  areas  of  strengths  or  developmental  needs.  This 
assessment  makes  it  possible  to  match  a  candidate's  specific  qualifi¬ 
cations  with  specific  job  requirements. 

The  immediate  supervisor  is  in  the  best  position  to  assess  the 
qualifications  of  a  subordinate.  However,  the  manager  selecting  an 
individual  for  a  position  must  assess  a  large  number  of  personnel  on 
the  basis  of  the  supervisors '  ratings.  The  selector  will  not,  in 
general,  be  familiar  with  the  specific  areas  of  strengths  or  develop¬ 
mental  needs  of  every  candidate.  Under  the  prior  system  the  ratings 
did  not  distinguish  a  profile  of  strengths  and  developmental  needs  in 
the  candidates  for  promotion. 


Purpose 

This  chapter  gives  some  background  to  the  basic  changes 
necessary  to  change  the  previous  rating  system  into  a  more  successful 
promotability  rating  system  for  ASD.  These  changes  were  in  (1)  the 
basis  of  rating,  (2)  the  nurber  of  scales,  (3)  the  degree  of  overlap, 
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(4)  the  scale  definitions,  (5)  the  rating  scale  position  anchors,  (6) 
the  coverage  of  job  requirements,  (7)  the  management  participation, 
and  (8)  the  training. 


Basis  of  Rating 

In  the  rating  system  prior  to  MPAS,  the  basis  of  the  ratings 
was  the  amount  of  supervision  required  by  the  subordinates  (18;  22). 
This  concept  is  not  desirable  because  it  does  not  address  the  central 
issue  of  (1)  the  present  effectiveness  or  (2)  the  promo tability  of 
the  subordinate. 

Uhder  the  MPAS  the  subordinate  was  rated  on  the  basis  of  his 
or  her  observable  job  performance  rather  than  personal  traits.  This 
method  is  more  reasonable  than  the  prior  system  because  the  amount  of 
supervision  required  by  subordinates  may  differ  for  reasons  that  are 
irrelevant  to  their  potential  for  promotion  (3;  4;  6;  9:745-775;  12). 

Number  of  Scales 

In  the  prior  rating  system,  the  supervisor  was  required  to  use 
twenty-eight  different  scales  or  dimensions  to  indicate  the  promot- 
ability  of  a  subordinate  (18;  22).  Under  the  MPAS  only  eleven  factors 
or  dimensions  are  used  for  technical  and  professional  subordinates  and 
eight  factors  for  clerical  personnel.  This  reduction  in  the  nuiber  of 
factors  has  the  advantage  of  reducing  not  only  the  difficulty  of  rating 
personnel  but  of  reducing  the  time  required  to  do  the  rating.  In 
addition,  there  is  evidence  that  five  to  eight  rating  scales  tend  to 
yield  better  quality  ratings  in  comparison  to  a  greater  number  of 
rating  scales  (7;  13;  16). 
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Lack  of  overlap  in  the  factor  scales  in  a  rating  system  is 
useful  because  it  allows  a  more  appropriate  promotion  decision  by  the 
selector  when  promotion  opportunity  arises.  Under  the  prior  system 
the  rating  scales  on  the  different  factors  exhibited  a  high  degree  of 
overlap.  These  rating  scales  yielded  promotional  profiles  that  did 
not  differ  from  one  employee  to  another  (18;  22). 

The  rating  format  used  in  the  MPAS  had  been  previously  tested 
in  other  organizations.  The  resulting  evidence  suggests  that  less 
overlap  occurs  in  this  type  of  rating  scale.  Training  the  raters  in 
the  use  of  the  rating  format  further  decreases  the  lack  of  overlap 
(11;  16;  17;  19;  21). 

Scale  Definitions 

In  the  prior  rating  system,  the  clarity  of  the  scale  defini¬ 
tions  varied  considerably.  Some  of  the  scales  were  clearly  defined 
and  some  were  ambiguous  (18;  22).  If  the  scale  definitions  are 
unclear,  the  rater  has  to  rely  on  an  overall  impression  of  the  per¬ 
formance  of  the  ratee  in  order  to  rate  the  ratee.  Lack  of  clarity 
of  the  scale  definitions  tends  to  produce  a  high  degree  of  overlap 
among  the  ratings  obtained  (8;  9:745-775;  19;  21). 

Rating  Scale  Position  Anchors 

The  prior  rating  system  defined  the  scale  steps  or  position 
anchors  by  adjective  descriptions  (18;  22).  The  trouble  with  this 
approach  is  that  it  requires  different  supervisors  to  interpret  the 
adjectives  for  themselves.  With  little  concrete  information  to  tie 


down  the  definition  of  each  adjective,  the  supervisors  will  tend  to 
interpret  the  scale  position  steps  by  their  own  standards. 

Under  the  MPAS,  general  global  descriptions  of  behavior  are 
used  to  define  the  low,  median,  and  high  rating  for  each  seven-step 
scale.  These  standard  descriptions  were  developed  by  supervisors 
writing  critical  incidents  which  they  felt  were  typical  of  the  job 
performance  of  subordinates  whose  ratings  average  six  or  more  over  the 
different  rating  dimensions.  This  procedure  was  less  precise  than  the 
method  used  by  Campbell,  et  al.,  (4;  6)  in  which  the  supervisor  is 
given  a  set  of  job-specific  performance  incidents  and  asked  to  deter¬ 
mine  which  incident  could  be  expected  from  the  subordinates.  The 
strength  of  Campbell 's  method  is  its  specificity.  The  high  degree  of 
specificity  required  by  the  method  makes  it  inapplicable  for  develop¬ 
ment  of  scales  intended  for  broad  application  to  many  complex  Jobs  at 
different  organizational  levels,  such  as  those  that  exist  in  the  ASD. 
In  addition,  the  highly  specific  method  used  by  Campbell  tends  to 
assume  prior  knowledge  of  job  requirements.  In  view  of  the  frequently 
unpredictable  nature  of  tasks  in  the  ASD,  the  more  global  approach  to 
the  definition  of  scale  steps  was  used  (1). 

The  MPAS  differs  from  the  prior  rating  format  in  that  it 
involves  much  more  careful  and  detailed  definition  of  the  scales  and 
their  position  anchors.  There  is  a  large  body  of  evidence  that  a  more 
detailed  or  structured  set  of  instructions  or  definitions  in  the 
rating  format  tends  to  reduce  the  difficulty  of  judgment  for  the 
rater  and  to  increase  the  quality  of  the  obtained  ratings  (9:745-775; 
16;  17;  19;  21). 
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Coverage  of  Job  Requirements 


In  the  prior  rating  system  the  rating  scales  were  not  clearly 
relevant  to  job  performance.  In  addition,  not  all  aspects  of  job 
performance  were  represented  by  the  current  set  of  rating  scales 
(18;  22).  Under  the  MPAS  the  rating  dimensions  were  defined  by  those 
individuals  who  would  be  using  the  system  to  make  ratings.  Thus,  the 
appropriateness  of  the  rating  scales  to  the  jobs  was  built  into  the 
system.  Those  who  are  most  qualified  to  design  the  rating  scales  were 
involved  in  designing  them.  The  fact  that  individuals  who  use  the 
system  had  input  to  its  design  has  the  effect  of  increasing  the 
relevance  of  the  ratings  (4;  6;  8;  10;  19). 

Management  Participation 

In  the  prior  rating  system  there  was  little  management  control 
over  the  rating  format  or  the  quality  of  the  ratings.  There  was 
little  evidence,  if  ary,  that  there  was  any  effective  feedback  to  the 
supervisors  on  the  quality  of  these  ratings.  Various  managers  gave 
the  impression  that  the  process  of  performing  the  merit  appraisal 
ratings  was  unimportant  and  a  waste  of  valuable  time  (18;  22). 

It  was  true  that  the  prior  system  with  its  apparent  high 
leniency  factor  created  no  inmediate  problem  for  the  supervisor  who 
performed  the  ratings.  The  difficulty  in  the  quality  of  the  ratings 
became  apparent  when  the  ratings  were  to  be  used  as  a  basis  for  pro¬ 
motion.  Cue  to  the  lack  of  a  discemable  difference  in  ratings 
between  candidates,  the  training  and  experience  factors  were  a  major 
determinant  of  the  promo tability  ratings.  This  is  unfortunate 
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because  the  immediate  supervisor  is  the  single  person  most  qualified 
to  assess  the  promo tability  of  a  subordinate.  The  supervisor's  judg¬ 
ment  is  potentially  of  much  greater  value  than  the  training  and 
experience  information  by  itself. 

Under  the  MPAS,  members  of  management  have  defined  the  rating 
format  and  scale  content.  This  is  an  improvement  over  the  prior 
system  because  it  insures  that  the  scale  content  will  more  adequately 
reflect  relevant  aspects  of  Job  performance.  This  total  management 
input  to  the  system  is  essential  if  management  is  to  gain  information 
from  the  ratings  which  is  useful  in  making  promotion  decisions  (16; 

18;  19). 


Training 

There  are  several  studies  (13;  16;  17;  19;  23)  which  indicate 
the  effectiveness  and  importance  of  training  raters  in  reducing 
leniency  and  other  rating  errors. 

In  the  prior  rating  system  there  was  very  little  in  the  way  of 
training  of  the  raters  in  how  to  make  ratings  (18;  22).  For  any  kind 
of  rating  method,  it  is  Important  for  the  rater  to  know  how  to  use  the 
rating  format  to  describe  the  Job  performance  of  the  ratee.  The  lack 
of  training  and  the  closely  associated  lack  of  standards  of  rating 
quality  left  the  interpretation  of  the  rating  scales  and  the  scale 
steps  or  anchors  up  to  the  individual  supervisor.  This,  in  turn, 
tended  to  produce  differences  in  the  way  the  ratings  were  performed 
by  different  supervisors.  Given  no  other  standards  of  performance  for 
the  ratings  and  their  perceived  low  priority,  the  supervisor  did  what 
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was  easiest  for  him  or  her  in  the  context  of  other  pressing  tasks. 

Poor  quality  and  inflated  ratings  were  the  results  (13;  17;  19;  23). 

It  would  not  be  practical  to  design  a  rating  system  which 
would  describe  everything  a  subordinate  might  do  in  his  or  her  Job. 
This  is  especially  true  if  the  job  in  question  involves  highly  com¬ 
plex  and  rapidly  changing  technology  with  somewhat  unpredictable  task 
requirements  (16).  This  situation  tends  to  be  true  at  ASD. 

Correct  rating  of  subordinates  is  necessarily  a  matter  of 
informed  judgment.  Under  MPAS,  extensive  training  programs  were 
implemented  to  ensure  a  higher  quality  of  ratings.  The  cases  used  in 
the  training  were  not  specific  to  a  particular  branch  because  their 
purpose  was  to  raise  questions  as  well  as  answer  them,  The  purpose  of 
the  cases  was  to  train  the  supervisors  in  how  to  match  a  specific 
subordinate's  performance  to  the  appropriate  scale  and  to  the  appro¬ 
priate  scale  anchor  (or  scale  step)  within  that  scale.  The  training 
gave  the  supervisor  a  chance  to  actually  practice  the  rating  of  sub¬ 
ordinates  in  the  form  of  the  case  studies  and  was  given  immediate 
feedback  as  to  the  correctness  of  his  or  her  rating.  This  technique 
of  training  raters  helped  to  correct  the  errors  such  as  first 
impressions  and  halo  effects  (13). 

Because  of  the  need  for  informed  judgment  and  for  training 
credibility,  the  trainers  used  vrere  line  managers  who  were  highly 
familiar  with  the  work  done  by  the  supervisors'  subordinates.  These 
managers  could  answer  questions  about  the  applicability  of  the  scales 
to  particular  situations  (1). 


Summary 


This  chapter  has  shown  the  major  elements  and  features  of  the 
promotion  appraisal  system  which  have  been  changed  with  the  develop¬ 
ment  of  the  MPAS.  How  successful  were  these  changes  in  creating  an 
effective  and  useful  tool  for  determining  the  promo tability  of  indi¬ 
viduals?  Did  the  MPAS  show  high  validity  and  acceptability  to  the 
employees?  These  questions  are  examined  in  the  following  chapters. 
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CHAPTER  3 


METHODOLOGY 

Purpose 

This  chapter  presents  the  methodology  used  to  achieve  the 
objectives  of  (1)  trend  analysis,  (2)  validity  analysis,  and  (3) 
acceptability  of  the  MPAS  to  employees . 

Trend  Analysis 

To  show  some  indication  of  effectiveness  of  the  implementa¬ 
tion  of  the  MPAS  and  to  show  any  tendency  for  the  MPAS  to  return  to  a 
condition  of  inflation  of  rating,  a  trend  analysis  of  the  distribu¬ 
tion  of  ratings  for  successive  years  from  1975  through  1980  was 
conducted.  Data  for  the  graph  of  the  previous  rating  system  were 
obtained  from  the  1978  MPAS  status  report  (1).  Data  and  rating  means 
for  the  1978  through  1980  graphs  were  obtained  from  listings  on  file 
at  ASD/DPCH. 

A  comparison  between  the  1975-70  graphs  and  the  first  MPAS 
graph  for  the  overall  rating  system  was  made  to  determine  the  effec¬ 
tiveness  of  the  initial  implementation.  A  decrease  in  the  percentage 
of  individuals  rated  at  the  high  end  of  the  rating  scale  would  tend 
to  indicate  a  more  effective  rating  system.  A  further  comparison  of 
histograms  and  rating  means  for  succeeding  years  was  made  to  obtain 
an  indication  of  a  return  to  inflated  ratings.  This  trend  would  be 
reflected  on  the  histograms  by  a  shift  toward  the  upper  end  of  the 
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rating  scale.  Further  evidence  of  inflated  ratings  would  be  a  pro¬ 
gressive  increase  in  the  yearly  rating  mean. 


Validity  Analysis 

A  second  technique  was  used  to  examine  the  validity  of  the 
overall  ratings  obtained  by  the  MPAS.  The  data  used  in  this  test  were 
obtained  in  June  of  1979  by  ASD/DPCH.  Four  different  studies,  as 
shown  in  Appendices  D,  E,  F,  and  G,  were  administered  for  eighty-eight 
different  target  employees  to  obtain  the  data.  Study  Number  Thirty 
was  administered  to  the  immediate  supervisors  of  the  target  employees . 
The  supervisors  were  advised  that  the  identity  of  the  target  employees 
would  remain  anonymous  and  the  data  would  only  be  used  for  validation 
of  the  MPAS.  The  supervisors  were  asked  to  provide  the  actual  ratings 
given  the  enployee  and  the  "precise"  ratings  they  would  have  fiw  the 
employee  if  no  outside  pressures  existed.  Study  Number  Thirty-One  was 
administered  to  the  employees  for  a  self-rating.  Study  Number  Thirty- 
Two  was  administered  to  a  peer  of  the  target  employee.  The  peer  was 
to  have  been  able  to  observe  the  target  employee  at  wrk  and  to  have 
some  knowledge  of  the  target  employee's  job.  Study  Number  Thirty-Four 
was  administered  to  an  alternate  supervisor  at  or  above  the  actual 
supervisor's  level  and  with  some  knowledge  of  the  target  employee's 
work  activities.  All  studies  were  to  be  returned  with  no  indication 
of  the  employee's  name  but  with  the  different  studies  linked  by  a 
random  number  common  to  each  form. 

The  data  obtained  from  these  studies  are  shown  in  Appendix  H. 
Note  should  be  made  of  the  great  quantity  of  missing  information  in 


this  data.  For  example,  only  fifty-six  of  the  eighty-eight  target 
cases  were  available  for  use  in  obtaining  pair-wise  Pearson  Correla¬ 
tion  coefficients  between  the  overall  actual  ratings  and  the  immediate 
supervisors  "precise"  overall  ratings.  When  computing  the  pain-wise 
Pearson  Correlation  coefficients  for  the  individual  rating  factor  of 
Speaking  Ability  in  the  self  rating  study,  only  twenty-four  of  the 
eighty-eight  cases  were  available. 

Since  all  employees  were  not  rated  on  the  same  number  of 
factors,  an  overall  rating  for  the  different  individuals  and  the 
different  studies  was  computed  by  multiplying  the  average  of  the 
factors  by  a  value  of  ten.  This  computation  gave  an  overall  rating 
that  ranged  from  ten  to  seventy.  This  technique  of  obtaining  a 
rating  is  the  same  technique  as  used  in  the  MPAS. 

These  overall  ratings  were  then  studied  for  correlation  using 
Pearson  Correlation  and  Student  t-test.  A  study  of  the  two  analysis 
outcomes  would  give  some  information  concerning  the  overall  validity 
of  the  ratings  obtained  in  the  MPAS. 

A  follow-up  technique,  using  the  same  data  obtained  above,  is 
the  multitrait-nultimethod  matrix  suggested  by  Campbell  and  Fisk  (5). 
This  method  was  used  to  determine  the  convergent  and  discriminate 
validity  of  the  factors  used  in  the  MPAS.  An  example  of  a  multi  trait- 
multimethod  matrix  is  shown  in  Table  3.1. 

To  indicate  convergent  validity  (the  amount  of  agreement 
between  raters  on  the  same  traits),  the  values  in  the  validity  diag¬ 
onals  should  be  significantly  different  from  zero  (5:82).  For 
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TABLE  3.1 


A  SYNTHETIC  MULTI  TRAIT-MLTLTIMETHQD  MATRIX 


Method  1  Method  2  Method  3 


Traits  Ax  B1  C ^  ^  B2  C2  *3  B3  C3 


Note:  =  Validity  Diagonal 

Heterotrait-Heteromethod  Triangle 
=  Heterotrait-Monornethod  Triangle 


example ,  all  of  the  validity  diagonals  in  Table  3.1  have  a  value 
significantly  higher  than  zero. 

Discriminate  validity  can  be  indicated  by  the  validity  diag¬ 
onal  value  being  higher  than  the  values  in  its  row  and  colurm  of  the 
heterotrait-heteromethod  triangles.  That  is,  the  validity  value  for 
a  variable  should  be  higher  than  the  correlations  between  that  variable 
and  ary  other  variable  not  having  either  a  common  trait  or  method 
(5:82).  For  exanple,  in  Table  3.1  the  validity  diagonal  for  Trait 
A^-Ag  (.56)  is  higher  than  the  values  for  Trait  A^-B^  (.23),  Trait 
A1-C3  (.11),  Trait  B^-Ag  (.22),  and  Trait  C^-Ag  (.11).  All  of  the 
validity  diagonals  in  Table  3.1  are  higher  than  the  values  in  their 
row  and  colurm  of  the  heterotrait-heteromethod  triangles. 

A  second  means  of  showing  discriminate  validity  involves 
comparing  the  validity  diagonal  value  with  the  values  in  its  row  and 
column  for  the  heterotrait-monomethod  triangles.  This  inplies  that 
the  correlations  are  higher  when  different  methods  are  used  to  measure 
the  same  variable  than  whan  different  variables  are  measured  by  the 
same  method  (5:83).  For  exanple,  in  Table  3.1  the  validity  diagonal 
for  Trait  B^-Bg  (.57)  is  higher  than  the  value  for  Trait  B^-C^  (.37) 
but  not  higher  than  the  value  for  Trait  Ag-Bg  (.68).  Therefore,  it 
would  be  questionable  whether  ary  discriminate  validity  was 
indicated. 

For  the  purposes  of  this  analysis,  the  rating  fora  factors 
will  be  used  as  traits  and  the  five  different  sources  of  ratings  will 
be  used  as  methods.  The  multi  trait-multimethod  matrix  technique  will 
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be  applied  to  the  three  separate  forms  used  in  the  MPAS  (Form  A, 
professional;  Form  B,  technical;  Form  C,  clerical).  Due  to  the 
varying  number  of  cases  used  to  compute  the  correlation  coefficients, 
the  normal  technique  of  corparing  the  magnitude  of  the  coefficients 
will  not  be  used.  Instead,  the  comparisons  will  be  made  between  the 
level  of  significance  of  each  correlation  coefficient. 


Acceptability  Analysis 

The  final  technique  used  to  analyze  the  MPAS  was  an  attempt  to 
determine  if  the  employees  have  accepted  the  MPAS  as  a  valid  means  of 
measuring  proootability.  The  analysis  was  dene  through  comparison  of 
the  results  of  a  September,  1978,  technical  report  (20)  and  similar 
questions  on  1979  questionnaires  administered  to  1,077  enployees  who 
were  rated  on  the  MPAS  and  371  supervisors  who  rated  enployees  on  MPAS 
forms.  Copies  of  these  questionnaires  are  shown  in  Appendices  I  and  J. 
The  corrparlson  was  based  on  questions  concerning  the  perception  of 
validity  and  the  overall  reaction  of  the  personnel  to  the  MPAS.  A 
description  of  the  March,  1979,  questionnaire  is  the  same  as  this 
description  of  the  1978  questionnaires  contained  in  the  AFIT  TR  78-5: 

Eleven  factors  (scales)  were  contained  in  the  new 
appraisal  system:  (1)  self  management;  (2)  work  adminis¬ 
tration;  (3)  problem  analysis;  (4)  decision  making;  (5) 
speaking  ability;  (6)  writing  ability;  (7)  working  rela¬ 
tionships;  (8)  work  leadership;  (9)  personnel  management 
and  EEO  conrnitment;  (10)  performance  under  pressure;  and, 

(11)  work  output.  The  questionnaire  used  to  evaluate  the 
new  appraisal  system  sought  four  types  of  information 
about  each  factor.  Specifically,  it  was  designed  to 
gather  supervisor  and  subordinate  S&E  opinions  concerning 
the  amount  of  information  available  related  to  each 
factor,  the  meaningfulness  of  each  factor,  each  factor's 


18 


appropriateness,  and  the  difficulty  in  understanding  or 
rating  each  factor.  Additionally,  questions  were 
included  which  requested  supervisor  and  subordinate  views 
of  specific  benefits  of  the  new  system,  as  well  as  an 
overall  evaluation  of  the  form  by  those  rated  with  it. 
Lastly,  write  in  consents  were  solicited  £?0:£7. 


Summary 

This  chapter  presented  the  methodology  used  to  analyse  trends 
toward  a  return  to  inflated  ratings.  The  methodology  for  analysis  of 
validity  and  for  the  acceptance  trend  of  MPAS  was  also  shown. 

Chapter  4  will  give  the  results  of  these  analyses. 


CHAPTER  4 


DATA  AND  ANALYSIS 

Introduction 

This  chapter  will  present  the  results  from  the  methodology 
described  in  Chapter  3.  First,  the  results  of  the  trend  analysis  and 
some  possible  reasons  for  the  variation  in  the  histograms  will  be 
presented.  The  results  of  the  validity  analysis  and  some  discussion 
of  the  findings  will  follow.  Finally,  the  results  of  the  study  for 
a  change  in  attitude  of  the  employees  and  supervisors  toward  the 
acceptability  of  the  MPAS  will  be  presented. 

Trend  Analysis 

When  the  histograms  of  the  appraisal  ratings  for  the  years 
1975  through  1977  (Figures  4.1  through  4.3)  are  studies  in  detail,  it 
becomes  evident  that  there  is  an  inprovement  in  distribution  of 
ratings  for  the  year  1977  (Figure  4.3).  Note  that  the  distribution  of 
the  1975  and  1976  ratings  in  Figures  4.1  and  4.2  are  highly  concen¬ 
trated  at  the  high  end  of  the  scale  compared  to  the  1977  ratings  in 
Figure  4.3,  which  are  much  closer  to  being  symmetrically  distributed. 
Since  the  data  represented  by  Figure  4.3  is  the  first  data  available 
under  the  MPAS,  it  seems  reasonable  to  say  the  initial  implementation 
of  the  MPAS  was  successful.  However,  there  was  an  additional  factor 
present  at  the  same  time  which  may  confuse  the  interpretation  of  the 
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histograms.  As  stated  in  the  1978  status  report  on  the  MPAS  (1),  the 
sharp  peak  Just  below  the  6.1  value  was  probably  influenced  by  the  use 
of  guidelines  in  1977.  This  use  of  guidelines  may  be  one  factor  pro¬ 
ducing  the  more  symmetrical  distribution  in  the  1977  ratings  in 
Figure  4.3 

A  visual  analysis  of  the  1978-80  histograms  for  engineers  in 
ASD  (Figures  4.4  through  4.6),  along  with  the  corresponding  rating 
means,  give  an  indication  that  the  ratings  obtained  from  the  MPAS  are 
showing  a  trend  toward  a  return  to  inflated  ratings.  The  distribu¬ 
tions  from  year  to  year  tend  to  be  skewed  more  and  more  negatively. 
This  means  that  a  higher  percentage  of  the  appraisal  ratings  are 
moving  toward  the  upper  end  of  the  rating  scale  each  year.  The  fact 
that  the  mean  of  the  ratings  increases  each  year  also  reinforces  this 
finding.  Additional  histograms  showing  the  distribution  of  appraisal 
ratings  for  the  years  1978  through  1980  are  presented  in  Appendix  K. 

In  every  case,  the  data  presented  indicates  a  return  to  inflated 
ratings  after  the  initial  improvement  upon  implementation .  The 
rating  means  presented  in  Table  4.1  also  give  evidence  that  in  every 
case  the  appraisal  ratings  are  moving  toward  inflated  values. 
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TABLE  4.1 


RATING  APPRAISAL  MEANS 


_ YEAR _ 

ORGANIZATION  1978  1979  1980 

N  MEAN  N  MEAN  N  MEAN 


ASD-Engineers 

1002 

61 

1002 

63 

989 

66 

ASD-Professional 

1411 

61 

1574 

63 

1639 

65 

ASD-Technical 

122 

59 

164 

62 

191 

64 

ASD-Clerical 

634 

55 

896 

63 

876 

65 

ASD-A11 

2183 

60 

2647 

63 

2713 

65 

4950th  Test  Wing 

813 

58 

914 

61 

899 

63 

Foreign  Technology  Division 

Air  Force  Wright 

310 

57 

426 

59 

499 

62 

Aeronautical  Laboratory 

Air  Force  Aerospace  Medical 

1749 

58 

1800 

60 

1753 

62 

Research  Laboratory 

Air  Force  Human  Resources 

124 

58 

126 

60 

123 

62 

Laboratory 

Joint  Aeronautical  Material 

20 

53 

6 

67 

25 

62 

Activities  (AFSC/PM) 

- 

- 

31 

61 

34 

62 

All  Serviced  Activities 

5244 

59 

5959 

61 

6059 

64 
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The  correlations  shown  in  Table  4.2  indicate  that  the  overall 


actual  ratings  were  most  highly  correlated  with  the  overall  ratings 
received  from  the  target  employee's  irrmediate  supervisor.  Even 
though  the  immediate  supervisors  had  been  informed  that  the  "precise" 
ratings  they  gave  would  not  be  used  for  anything  other  than  an 
analysis  of  the  MPAS,  it  is  probably  true  that  they  still  felt  a 
considerable  amount  of  pressure  to  report  the  sane  ratings  as  were 
recorded  during  the  actual  rating  cycles.  Therefore,  the  high  corre¬ 
lation  between  the  overall  actual  rating  and  the  supervisors'  overall 
"precise"  rating  is  not  surprising.  There  were  three  notable  excep¬ 
tions  to  the  supervisors'  overall  rating  being  the  most  highly 
correlated.  For  female  employees  on  all  forms,  the  highest  correla¬ 
tion  was  between  the  actual  overall  ratings  and  the  overall  alternate 
supervisor  ratings.  Female  clerical  (Form  C)  enployees  and  male 
technical  (Form  B)  enployees  show  the  highest  correlation  between  the 
actual  overall  ratings  and  the  overall  ratings  received  from  their 
fellow  employees.  If  the  high  correlation  shown  by  the  irrmediate 
supervisor  is  ignored  by  reason  of  the  expected  relationship,  the 
alternate  supervisors'  overall  ratings  show  the  highest  correlation 
for  all  forms  and  sexes.  The  alternate  supervisors'  overall  ratings 
have  the  highest  or  close  to  the  highest  correlation,  other  than 
immediate  supervisor,  in  eight  of  the  nine  different  categories  shown. 

For  the  Form  C  (Technical)  female  sanple,  it  was  interesting 
to  find  the  highest  correlation  with  the  Peer  method  of  measurement. 
The  reason  for  this  may  be  the  small  number  of  cases  in  this  category. 
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TABLE  4.2 


TABLE  OF  PEARSON  CORRELATIONS  AND 
STUDENT  t-TEST  VALUES 


FORM 

SEX 

ACTUAL  OVERALL 
RATINGS  AGAINST 

#  OF 
CASES 

CORRELATION 

COEFFICIENT 

2-TAILED 

PROBABILITY 

Both 

Supervisor 

56 

.783* 

.018® 

Self 

35 

.405* 

.363 

Peer 

49 

.509* 

.039® 

Alt-Supervisor 

37 

.573* 

.258 

Female 

Supervisor 

21 

.599* 

.132 

Self 

14 

.066 

.392 

Peer 

16 

.527** 

.191 

A1 t-Superviscr 

17 

.601* 

.704 

Male 

Supervisor 

35 

.887* 

.053 

Self 

21 

.544* 

.704 

Peer 

33 

.512* 

.121 

Alt-Supervisor 

20 

.536* 

.254 

Both 

Suoervisor 

31 

.891* 

.033® 

Self 

17 

.339 

.827 

Peer 

30 

.452* 

.124 

Alt-Supervisor 

20 

.622* 

.066 

Female 

Supervisor 

2 

1.000* 

.500 

Self 

2 

-1.000* 

.647 

Peer 

3 

.598 

.198 

Alt-Supervisor 

3 

.963** 

.081 

Male 

Suoervisor 

29 

.886* 

.039® 

Seif 

15 

.548** 

.762 

Peer 

27 

.458* 

.255 

Alt-Supervisor 

17 

.556* 

.155 

Male 

Supervisor 

5 

.925* 

.853 

Self 

5 

.272 

.915 

Peer 

5 

.970* 

.294 

Alt-Supervisor 

3 

.268 

.660 

Both 

Supervisor 

20 

.578* 

.145 

Self 

13 

.420 

.063 

Peer 

14 

.553* 

.316 

Alt- Supervisor 

14 

.571** 

.907 

Female 

Supervisor 

19 

.575* 

.145 

Self 

12 

.399 

.074 

Peer 

13 

.589** 

.437 

Alt-Supervisor 

14 

.571** 

.907 

Note':  No  female  Form  B  (Technical)  sample.  Not  enough  male  cases  In 
Form  C  (Clerical)  sample  to  generate  values. 

©Reject  null  hypothesis  of  equal  mean  ratings. 

*p  *  0.01 
**p  *  0.05 
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Almost  without  exception,  the  method  of  measurement  that 
exhibited  the  lowest  correlation  with  the  actual  overall  ratings  was 
the  Self  method.  Furthermore,  the  Self  method  of  rating  measurement 
showed  the  least  nunfcer  of  cases  to  compare  to  the  actual  ratings. 

One  hypothesis  for  these  results  is  that  the  target  individuals  may 
not  have  felt  the  follow-up  studies  would  be  of  any  use  in  making  the 
MPAS  more  consistent  with  their  perceptions.  Therefore,  some  did  not 
bother  to  respond  to  the  questionnaires. 

For  the  Student  t-test  values,  the  null  hypothesis  being 
tested  was  that  the  difference  between  the  means  of  the  overall  ratings 
was  zero  .  The  alternate  hypothesis  was  that  the  differ¬ 

ence  between  the  means  of  the  overall  ratings  was  not  zero 

The  values  of  alpha  used  for  this  thesis  was  0.05. 

Using  the  hypotheses  and  alpha  levels  stated  above,  it  can  be  seen 
from  Table  4.2  that  most  of  the  probabilities  were  above  the  specified 
alpha  level.  Therefore,  the  null  hypothesis  of  equal  means  was 
accepted  for  most  cases.  Three  of  the  four  notable  exceptions  are  the 
probabilities  for  the  supervisor  ratings  in  the  sections  that  included 
all  respondents,  the  section  for  Form  A-All,  and  the  section  for 
Form  A-Males .  In  all  three  of  these  cases,  the  mean  of  the  "precise" 
ratings  was  higher  than  the  mean  of  the  actual  ratings.  The  fourth 
exception  is  for  the  peer  ratings  in  the  section  for  all  respondents. 

In  this  case,  the  peer  rating  mean  was  lower  than  the  actual  rating 
mean. 

For  the  Pearson  Correlations  to  effectively  indicate  any 
validity  for  the  MPAS,  the  correlations  must  not  only  be  relatJ  ?ly 


31 


high  but  the  means  for  the  different  methods  of  measurement  should 
statistically  be  the  sane.  From  Table  4.2,  the  data  gathered  from 
Alt-Supervisors  is  strongest  in  developing  validity  confidence  for 
the  MPAS  ratings.  The  correlation  is  high  and,  statistically,  the 
mean  ratings  are  the  same. 

Since  it  was  probable  that  the  "precise”  ratings  obtained 
from  the  immediate  supervisors  were  contaminated  by  the  perceived 
pressures  that  existed,  the  data  for  the  multi trait-multimethod 
matrix  was  obtained  using  only  the  four  methods  of  actual  rating, 
self  rating,  peer  rating,  and  alternate  supervisor  rating. 

The  results  with  respect  to  comparison  of  validity  values 
with  other  heteromethod  values  in  each  block  of  the  mul ti trait- 
mil  time  thod  matrix  method  for  convergent  and  discriminate  validity 
are  presented  in  suimarized  form  in  Table  4.3.  Table  4.3  values 
(Form  A-All)  are  condensed  from  the  data  given  In  Appendix  L, 
Although  all  combinations  of  sex  and  form  were  attenpted  on  the 
eighty-eight  cases  of  data  in  Appendix  H;  due  to  missing  data,  the 
information  on  Form  A-All  was  the  only  combination  that  generated 
output  with  all  correlation  coefficients  calculated.  Therefore, 
analysis  is  presented  cn  these  data. 

In  terms  of  convergent  validity,  Table  4.3  indicates  that 
significant  validity  was  achieved  in  five  of  the  eleven  factors.  The 
validity  values  for  Work  Administration,  Decision  Making,  Speaking 
Ability,  Writing  Ability,  and  Work  Leadership  did  show  significantly 
high  coefficients  in  at  least  half  of  their  validity  diagonals. 
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TABLE  4.3 — Continued 


In  terms  of  discriminate  validity.  Table  4.3  indicates  three 
factors  that  definitely  show  significant  validity.  Speaking  Ability 
shows  the  highest  significance  with  its  validity  values  having  higher 


significance  in  124  of  the  132  comparisons.  The  other  two  factors 
which  showed  high  discriminate  validity  were  Writing  Ability  and  Work 
Administration.  Table  4.4  contains  the  summarized  information  for  the 
heterota^t-mcnomethod  comparison  for  the  matrix  for  Form  A  in 
Appendix  L.  The  only  factor  which  indicated  a  significant  degree  of 
discriminate  validity  in  the  conparison  of  validity  diagonals  to  the 
values  in  the  rows  and  column  entries  of  the  heterotrait>HTiononethod 
triangles  was  Speaking  Ability. 

Considering  all  of  the  information  available  for  the 
rrultitraitwrultimethod  matrix,  Speaking  Ability  showed  the  strongest 
case  for  being  considered  a  validated  MPAS  rating  factor.  To  a  lesser 
degree,  Writing  Ability  and  Work  Administration  can  be  established  as 
validated  factors.  The  rest  of  the  factors  on  Form  A  (Professional) 
were  not  strongly  validated  by  this  data. 

Acceptability 

Since  the  acceptance  of  the  MPAS  by  the  employees  would  be 
reflected  in  the  extent  that  the  personnel  believed  the  MPAS  was  a 
valid  measurement  of  their  promotability,  the  responses  to  questions 
from  follow-up  questionnaires,  Studies  Twenty-Four  and  Twenty-Six, 
were  analyzed.  In  March  of  1979,  Study  Twsnty-Four  was  administered 
to  371  supervisors  and  Study  Twenty-Six  was  administered  to  1,077 
employees  rated  on  the  MPAS  rating  forms.  Copies  of  these  studies  are 


shown  in  Appendices  I  and  J.  Table  4.5  shows  the  responses  to  ques¬ 
tions  that  addressed  the  respondents'  perception  of  the  validity  of 
the  MPAS.  Validity  in  this  case  means  that  the  rating  system 
measures  what  it  was  intended  to  measure.  Table  4.5  also  shows  a 
conparison  of  the  responses  to  similar  questions  on  a  1978  follow-up 
questionnaire.  Since  the  1978  questionnaires  were  administered  at 
separate  times  to  AFWAL  and  ASD,  the  results  shown  for  the  1978 
questionnaire  were  computed  for  all  respondents  at  one  time.  For 
exarrple,  since  the  AFIT  Technical  report  (20)  shows  that  fifty-nine 
percent,  or  forty-one,  of  the  supervisor  respondents  for  ASD  chose 
Personnel  Management  and  EEO  Commitment  as  difficult  to  rate  and 
twenty-six  percent ,  or  twenty- four,  of  the  supervisor  respondents 
for  AFWAL  indicated  the  same  factor,  Table  4.5  indicates  that  forty 
percent,  or  65  divided  by  161  times  100,  of  the  total  supervisor 
respondents  indicate  Personnel  Management  and  EEO  ConrnitJnent  as 


difficult  to  rate 


TABLE  4.5 


PERCEIVED  VALIDITY 


Question  14  (Study  26).  Which  two  factors  did  you  find  most  difficult 
to  understand  in  terms  of  why  you  received  a  given  rating? 

Question  48  (Study  24).  Which  two  factors  did  you  find  most  difficult 
to  rate? 


A. 

3. 

C. 

D. 

E. 

F. 

G. 

H. 

I. 

J. 

K. 


%  Indicating  Factor 


1978  1979 

S&E  Supervisor  S&E  Supervisor 


Self  Management 

11 

9 

9 

13 

Work  Administration 

13 

20 

8 

8 

Problem  Analysis 

11 

12 

7 

10 

Decision  Making 

13 

14 

6 

10 

Speaking  Ability 

10 

6 

5 

3 

Writing  Ability 

11 

3 

5 

3 

Working  Relationship 

12 

2 

7 

4 

Work  Leadership 

Personnel  Management  & 

15 

15 

5 

6 

EEO  Commitment 
Performance 

18 

40 

13 

15 

Under  Pressure 

11 

7 

9 

8 

V/ork  Output 

10 

6 

7 

6 

Question  25  (Study  26).  In  your  experience,  do  you  think  the  Merit 
appraisal  ratings  were  performed  with  conscientious  use  of  the  factor 
definition  and  rating  steps  in  the  new  form? 


%  Indicating  Response 
1978  1979 

1/2  Definitely  not/Probably  not  40  30 

3/4  Probably  yes/De finitely  yes  60  69 

Question  26  (Study  26).  Based  on  the  wording  of  the  scale  steps,  do 

you  think  the  ratings  you  received  on  the  new  merit  appraisal  form 
should  have  been: 


%  Indicating  Response 


1978 

1979 

1/2 

Much  lower/Lower 

7 

6 

3 

Same 

55 

55 

4/5 

Higher/Mich  higher 

37 

39 

39 


TABLE  4.5 — Continued 


Question  27  (Study  26).  Compared  to  what  you  think  other  people’s 
ratings  may  have  been,  do  you  think  the  ratings  you  received  on  the 
new  merit  appraisal  form  should  have  been: 


%  Indicating  Response 
1978  1979 


1/2 

Much  lower/Lower 

2 

1 

3 

Same 

50 

47 

4/5 

Higher/Much  higher 

4 8 

52 

In  both  the  1978  and  1979  studies,  the  one  factor  that  the 
employees  indicate  is  difficult  to  understand  or  rate  is  the 
Personnel  Management  &  EEO  Commitment  factor.  All  of  the  factors  for 
employees  rated  on  the  forms  decrease  in  percentage  from  1978  to  1979. 
This  general  decline  seems  to  indicate  that  the  employees  were  gaining 
knowledge  of  and  confidence  in  the  system. 

Analysis  of  the  1979  responses  to  questions  25,  26,  and  27 
reveals  that  approximately  one-third  to  one-half  of  the  employees 
doubted  the  correctness  of  the  ratings  they  received.  These  percent¬ 
ages  are  almost  identical  to  the  1978  responses. 

Table  4.6  presents  the  results  of  the  responses  to  questions 
on  the  1979  follow-up  questionnaires  pertaining  to  the  overall 
reaction  of  the  respondents  to  the  MPAS.  Also  included  in  Table  4.6 
are  the  results  of  the  response  to  similar  questions  on  the  1978 
follow-up  questionnaires. 


TABLE  4.6 


OVERALL  REACTION 


Question  23  (Study  26).  How  much  irrpact  do  you  think  numerical  ratings 
given  on  the  appraisal  form  have  on  your  personal  progress? 

Question  37  (Study  24).  How  much  irrpact  do  you  think  numerical  ratings 
given  on  the  appraisal  forms  have  on  whether  an  employee  is  promoted? 


%  Indicating  Response 


1978 

1979 

S&E  Supervisor 

S&E  Supervisor 

1/2 

Very  little/Little 

55 

44 

49 

31 

3 

Some 

30 

46 

30 

37 

4 

A  great  deal 

15 

9 

21 

31 

Question  32  (Study  26).  Taken  on  the  whole,  what  is  your  reaction  to 


the  new  merit  promotion  appraisal  form? 

%  Indicating  Response 

1978 

1979 

1. 

I  dislike  the  new  form. 

9 

4 

2. 

I  have  a  somewhat  negative  reaction 
to  the  new  form. 

27 

11 

3. 

I  feel  neutral  about  the  new  form. 

52 

4. 

I  have  a  somewhat  positive  reaction 
to  the  new  form. 

55 

28 

5. 

I  am  very  positively  impressed  by 
the  new  form. 

8 

4 

In  both  the  1978  and  1979  studies,  the  supervisors  seemed  more 
Impressed  with  the  impact  of  the  rating  forms  than  the  employees  rated 
on  these  forms.  But  the  enployees  are  becoming  more  aware  of  the 
importance  of  the  MPAS  ratings.  This  fact  is  indicated  by  the  increase 
in  the  percentage  of  enployees  who  indicated  the  ratings  on  the  forms 
would  bhve  an  effect  on  their  progress. 
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The  decrease  in  the  percentage  of  respondents  that  indicated 
an  overall  negative  reaction  to  the  merit  promotion  appraisal  form  in 
Question  32  may  be  attributed  to  the  addition  of  the  added  response 
of  neutrality.  Response  number  three  may  have  absorbed  some  of  the 
borderline  negative  reactions  as  well  as  die  borderline  positive 
reactions.  In  1979  questionnaires,  the  extreme  responses  of  number 
one  and  five  are  approximately  half  the  percentage  of  responses  for 
1978  questionnaires. 

Information  obtained  from  Question  32  has  a  questionable 
value  in  determining  the  acceptability  of  the  MPAS  to  employees.  The 
responses  to  the  questions  in  Tables  4.5  and  4.6  indicated  that  the 
employees  are  becoming  more  aware  of  the  importance  of  the  MPAS  to 
their  personal  advancement  and  are  gaining  more  knowledge  of  the  MPAS 
and  its  forms.  The  responses  to  Questions  25  through  27  indicate 
there  is  still  some  doubt  that  ratings  received  were  correct. 

Summary 

The  analysis  of  the  results  of  this  research  indicates  that 
the  initial  implementation  of  the  MPAS  did  have  the  effect  of 
increasing  the  variance  in  the  ratings.  But  over  the  intervening 
years,  the  ratings  have  steadily  returned  to  an  inflated  status. 

Further  analysis  indicated  that  even  though  there  is  some 
indication  of  the  validity  of  the  overall  ratings  and  the  factors  used 
in  the  MPAS,  the  amount  of  missing  information  in  the  data  files 
causes  considerable  risk  in  interpretation  of  the  results. 


Finally,  the  analysis  of  the  results  of  follow-up  question¬ 
naires  indicated  that  the  enployees  rated  under  the  MPAS  were  accumu¬ 
lating  a  greater  understanding  of  the  MPAS,  both  the  forms  and  their 
uses.  Indication  was  still  noticed  of  an  uneasiness  as  to  the 
correctness  of  ratings  received  by  the  enployees. 

The  analysis  of  Chapter  4  has  included  specific  conclusions 
for  each  subobjective  as  each  was  addressed.  Chapter  5  contains  a 
surnnary  of  the  more  significant  conclusions,  as  well  as  recommenda¬ 
tions. 
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CHAPTER  5 


CONCLUSIONS  AND  RECOMMENDATIONS 

Overview 

In  the  preceding  chapters  the  subject  of  the  strengths  and 
weaknesses  of  the  MPAS  was  addressed.  Some  sub-objectives  were  stated 
as  research  goals.  The  plan  to  research  answers  was  then  described. 
The  results  of  the  research  plan  were  presented  and  analyzed.  This 
chapter  summarizes  the  findings  and  conclusions  of  this  research  and 
offers  recommendations  for  future  actions. 

Conclusions 

The  trend  analysis  indicates  a  definite  increase  in  variance 
— less  inflated  ratings — upon  implementation  of  the  MPAS.  The  picture 
at  inplementation  was  clouded  by  the  unofficial  guidelines  for  rating 
over  the  same  period  of  time.  Trend  analysis  also  shows  that  the  MPAS 
returned  to  inflated  ratings  over  a  tvro  to  three  year  time  period. 

This  trend  toward  inflated  ratings  is  probably  caused  by  two  factors: 
(1)  pressure  felt  by  supervisors  to  give  their  employees  a  high  rating 
to  make  them  competitive  for  jobs  and  (2)  lack  of  feedback  to  raters 
as  to  whether  their  performance  as  raters  is  acceptable. 

The  validity  of  the  overall  ratings  was  confirmed  to  a  degree. 
Also  the  convergent  and  discriminate  validity  of  sane  of  the  factors, 
most  notably  the  Speaking  Ability  factor,  was  strongly  supported. 
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Missing  data  from  the  data  base  for  the  validity  research  was  a  big 
problem  in  the  interpretation  of  the  results. 

Hie  final  major  conclusion  was  that  the  employees,  as  could 
be  expected,  are  gaining  more  insight  into  the  workings  of  the  MPAS. 
There  were  indications  from  the  research  on  the  1978-79  follow-up 
questionnaires  that  the  perceived  validity  (does  it  measure  what  it 
says  it  will  measure)  of  the  MPAS  has  been  on  the  increase.  Hiere 
were  also  indications  that  a  large  nunber  of  the  employees  still  do 
not  accept  the  rating  received  under  the  MPAS  as  correct,  a  finding 
which  is  perhaps  not  unexpected. 

Recommendations 

To  perform  a  n»iltithait-multimethod  matrix  analysis  of  the 
convergent  and  discriminate  validity  of  the  rating  factors  of  the 
MPAS,  the  necessary  questionnaires  (ones  very  similar  to  the  ones  in 
Appendices  D,  E,  F,  and  G  used  to  collect  data  for  this  research) 
should  be  administered  to  a  larger  cross  section  of  the  work  force  at 
ASD.  In  addition,  a  very  close  follow-up  of  the  questionnaires 
should  be  used  to  ensure  less  missing  data  for  research  purposes. 

The  value  of  the  whole  approach  depends  upon  an  adequate  sanpling  of 
the  target  employees.  Follow-up  research  using  the  more  adequate 
data  base  obtained  from  the  questionnaires  and  the  nultitrait- 
multimethod  matrix  analysis  would  be  very  useful  in  determining  if 
the  individual  rating  factors  in  the  MPAS  are  valid.  Also,  a  more 
adequate  data  base  would  enable  the  analysis  of  the  technical 
(Form  B)  and  clerical  (Form  C)  forms  for  convergent  and  discriminate 
validity. 
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Sunnary 

Even  though  this  research  was  somewhat  inconclusive,  some 
useful  information  was  obtained.  To  effectively  operate  a  corrplex 
organization  like  ASD,  it  is  imperative  there  be  an  effective  instru¬ 
ment  to  measure  job  performance  and  promotability.  Once  a  measuring 
instrument  acceptable  and  valid  to  employees  has  been  proven  effec¬ 
tive,  improved  management  of  our  most  precious  of  resources  (people) 
can  be  accomplished.  The  /1PAS  was  designed  to  accomplish  this  purpose 
and  was  apparently  successful  in  the  first  year  or  two  of  irrplementa- 
ticn.  It  appeared  to  have  a  fair  level  of  acceptance  and  validity. 

But  the  downfall  of  so  many  appraisal  systems — inflationary 
pressures — appears  to  have  seriously  weakened  the  MPAS. 


APPENDIX  A 

MERIT  PROMOTION  APPRAISAL  SYSTEM 
RATING  FORM  A  (PROFESSIONAL) 
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MERIT  PROMOTION  APPRAISAL  SYSTEM 

FORM  * 


0 AT C  or  APPRAISAL 

pepjoo  or  i 

SUPERVISION 

rnoM 

TO 

NAUt/ TITLE  0^  IMMEDIATE  SUPERVISOR 

NAME-  TITLE  Or  REVIEWING  SUPERVISOR 

INSTRUCTIONS  FOR  COMPLETING  SUPERVISORY  APPRAISAL 
KFTl'RS  TO  ASP/QPCE 

The  Air  Force  Merit  Promotion  Program  requires  the  immediate  or  first  level  supervisor  to  complete  an  appraisal  on 
each  employee  supervised.  This  Supervisory  Appraisal  system  has  been  designed  to  help  the  supervisor  make  and 
record  the  required  merit  promotion  appraisal. 

Steps  re  romptehmg  the  tom: 

1.  Supervisor  enters  the  required  data  on  the  top  of  this  page. 

2  Si v'-'tvisor  reviews  each  factor  and  determines  the  oroper  rating  as  discussed  in  the  next  section. 

Circle  the  proper  step  in  the  box  on  page  *2.  Circle  N/A  if  s  rating  factor  is  not  applicable. 

3.  Hive  the  appraisal  reviewed  by  the  next  higherlevel  supervisor,  resolve  differences;  both  supervisors 
srt  r  and  date  the  form  in  the  space  provided  on  page  a  2. 

4.  SI  iw  and  discuss  the  appraisal  with  the  employee;  have  the  employee  also  sign  on  page 

5.  If  the  employee  does  not  already  have  a  copy  of  ASD  Form  3S7,  please  give  them  onn. 

6.  Complete  the  card  punch  data  required  as  noted  in  the  instructions  on  page  *  1 

7.  Tear  off  pages  1  and  2  and  send  to  ASO/DPCE,  unless  given  other  instructions. 

Re/i ug  Eapipyer  Potenliei: 

Each  of  the  eleven  Merit  Promotion  Appraisal  factors  consists  of  three  parts: 

1.  A  basic  factor  definition  found  in  die  tirst  sentence  of  the  definition; 

2-  Suvfartnrs  in  the  remninderof  the  definition  which  represent  serivir.es  related  in  the  hssie  faemr  defini¬ 
tion;  end 

3.  A  seven  step  reting  scilc. 

As  an  example,  consider  Fsctor  1  -  Self  Management.  This  factor  relates  to  how  well  the  employee  organises, 
schedules,  and  accomplishes  own  work.  Self  Management  includes  such  subfactois  as  the  use  of  resources, 
setting  priorities,  coping  with  change,  assuming  responsibility,  and  amount  of  supervision  requited.  Some 
assignments  may  not  allow  the  employee  to  display  performance  in  ail  five  of  these  subfactois.  Soma  work 
assignments  may  involve  performance  refared  to  Self  Management,  but  not  specifically  coveted  by  any  of  the 
subfactois.  The  supervisor  must  decide  which  aspects  of  the  employee's  performance  ate  in  the  basic  factor 
definition  and  should  be  included  in  tba  rating. 

The  supervisor's  rating  of  rhe  employee’s  potential  must  be  baaed  on  his  ot  her  knowledge  of  the  employee's 
current  performance.  In  accordance  with  CSC  regulations,  the  Merit  Promotion  Appraisal  factors  have  been 
carefully  designed  to  focus  on  elements  of  the  employee's  job  which  are  important  lor  success  in  highewlevei 
work. 

When  rating  the  employee’s  potential,  the  supetviaot  should  give  appnpriate  consideration  to  the  complexity 
and  diversity  of  the  work  required  at  the  employee's  grade  level.  For  example,  proficiency  at  work  of  a  partic¬ 
ular  level  of  complexity  and  diversity  might  indicate  high  potential  lot  a  G.S.  7  Engiaaer.  However,  this  aaaae 
level  of  proficiency  would  indicate  low  potential  for  the  job  of  a  G.S.  13  Engineer. 

The  task  of  the  supervisor,  in  using  the  Merit  Promotion  Appraisal  Factors,  is  to  rata  on  a  scale  of  seven  steps 
the  employee's  potential  lot  success  in  higher  level  work.  Scale  steps  One,  Four  and  Sevan  pwvida  examples 
of  low,  average  and  high  1  avals  of  performance  for  some  of  the  subfactor*  Steps  Two,  Three.  Five  end  Six 
represent  intaimedtate  degrees  of  potential. 

Continuing  with  Factor  1  -  Self  Management,  at  m  example; 

Step  1.  Low  Potential  -  The  employee  ia  abla  to  organize  and  schedule  routine  wotfc  adequately,  hut  ueually 
requites  supervisory  assistance  m  organizing  more  difficult  won  end  in  setting  pnorities.  Supervisory 
intervention  i>  required  to  insure  either  completion  of  the  effort  ot  stopping  the  effort  before  the 
poiet  of  diminishing  returns. 

Continued  on  page  3 
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atep  4.  Average  Potential  -  The  tmployee  fa  adapt¬ 
able  to  changing  conditions  and  un foreseen 
events  in  organizing  and  scheduling  routine 
work.  The  supervisor  occasionally  aaaiats  the 
employee  in  planning  for  more  difficult  wotk. 
The  employee  takes  considerable  initiative  for 
managing  own  final  work  through  completion. 

Step  7.  High  Potential  -  The  employee’s  organization 
of  work  always  results  in  the  moat  efficient  use 
of  time.  Is  very  adaptable  to  constantly  chang¬ 
ing  workloads  and  unpredictable  contingencies, 
adjusting  priorities  accordingly,  and  the  super¬ 
visor  is  never  required  to  revise  the  employee’s 
plans.  Assumes  full  responsibility  for  all 
aspects  of  assigned  tasks,  assuring  that  the 
output  meets  the  needs  of  the  requestor. 

One  of  two  approaches  can  be  taken  in  rating  an  em¬ 
ployee  on  a  given  factor.  One  approach  is  to  consider 
the  factor  as  a  whole  and  rate  the  employee’s  performance 
relative  to  the  basic  factoi  definition  based  on  your  over¬ 
all  impression.  Using  this  approach,  you  would  decide 
between  a  rating  at  Step  2  and  e  rating  at  Step  3  by  simply 
looking  at  Step!  1  and  4  and  deciding  whether  the  em¬ 
ployee  is  ciotei  t  j  being  just  above  Step  1  or  just  below 
Step  4. 

A  second  approach  is  to  rate  the  employee  on  each  of  the 
subfactors  whic  i  anply  and  then  average  these  ratings. 
Let  us  consider  i  ery  simplified  example  of  how  two 
employees  migh  of  rated  on  Self  Management  using  this 
approach.  The  first  employee  is  reasonably  adaptabla 
to  changing  coniitions  but  has  nevet  produced  a  finished 
product  without  major  supervisory  intervention.  The 
second  employ-  e  is  reasonably  adaptable  to  changing 
conditions,  has  never  needed  s  push  to  get  the  job 
finished,  but  often  times  needs  assistance  in  deciding 
which  of  several  tasks  deserves  most  of  his  or  het 
attention.  The  t  »u  employees  would  be  rated  as  follows: 


Employee  / 

Employee  Behavior  Seale  Step 

Reasonably  adaptable  to  changing  conditions  4 
Naver  produced  a  finished  product  without  major 
supervisory  intervention  1 

Average  2  or  3 

Employee  II 

Employee  Behavior  Scale  Step 

Reasonably  adaptable  to  changing  conditions  4 
Never  needed  a  push  to  get  the  job  finished  7 

Often  needed  assistance  in  deciding  which  of 
several  tasks  deserved  most  of  his  or  her 
attention  3 

Average  4  or  5 


As  noted  earlier,  the  performance  rated  may  include 
aspects  related  to  the  batic  factor  definition,  but  not 
specifically  covered  by  any  of  tha  subfactors. 

Whichever  rating  method  it  used,  the  supervisor  must 
arrive  at  a  tingle  rating  fot  each  factor  which  represents 
his  or  hat  best  judgement  or  the  employee's  potential  for 
success  in  higher  level  work,  relative  to  each  basic 
factor  definition.  Furthermore,  the  supervisor  must  be 
prepared  to  explain  to  tha  employee,  using  specific 
example*  of  the  employee’s  job  performance,  why  the 
employee  was  rated  at  be/  she  was, 

»so  roan  sat 


FACTOR  1  -  SELF  MANAGEMENT 

This  factor  evaluates  the  employee’s  potential  for  pro¬ 
motion  baaed  upon  how  well  the  employee  organises, 
schedules,  and  accomplishes  his/her  own  Work.  Self  man- 
agemem  includes  such  areas  as  the  use  of  available 
resources;  setting  priorities  among  activitiaa;  coping 
with  changing  conditions  and  unforeseen  events;  assum¬ 
ing  responsibility  fot  the  completion  of  one's  owe  work; 
and  the  amount  supervision  required. 

STEPS 


7- 


6- 


5- 


H1GH  POTENTIAL  -  The  employee's  organiza¬ 
tion  of  work  always  results  in  tha  most  officiate 
use  of  time.  Is  very  adaptable  to  constantly 
changing  workloads  and  unpredictable  coatingea- 
cies,  adjusting  priorities  accordingly,  and  tha 
supervisor  is  never  required  to  revise  the  employ¬ 
ee’s  plana.  Assumes  full  responsibility  for  all 
aspects  of  assigned  tasks,  assuring  that  the 
output  meets  the  needs  of  tha  requestor. 


4- 


3- 


2- 


1- 


AVERAGE  POTENTIAL -The  employeeis  adapt¬ 
able  to  changing  conditions  and  unforeseen 
events  in  organizing  and  scheduling  routine  work. 
1  ne  supervisor  occasionally  assists  tne  employee 
in  planning  for  more  difficult  work.  Tbo  employee 
taken  considerable  initiative  for  managing  own 
final  work  through  completion. 


LOW  POTENTIAL  -  The  employee  in  Sbla  to 
organise  and  schedule  routine  work  adequately, 
but  usually  requires  supervisory  aasistaaca  la 
organizing  more  difficult  work  and  in  setting 
priorities.  Supervisory  intervention  is  required 
to  insure  eithet  completion  of  the  effort  or  Mopping 
the  effort  before  the  point  of  diminishing  rstnrns. 


ease  >  or  a  naan 
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FACTOR  :  -  WORK  ADMINISTRATION 
(Supannsota/ilanagara  and  other  designated  employee*’) 

This  (actor  evaluates  the  employee’s  potential  for  pro* 
motion  based  upon  how  well  the  employee  organizes  and 
schedules  the  work  of  others.  Work  administration  in¬ 
cludes  such  areas  as  handling  the  impact  of  new  and 
unanticipated  work  on  present  work  and  coping  with 
changing  conditions  end  unfoteseen  events;  establishing 
goals  and  milestones  for  others  and  delegating  authority; 
tracking  and  managing  positions,  manpower,  and  other 
resources;  establishing  and  changing  work  priorities;  and 
assuming  responsibility  for  the  work  of  others. 


FACTOR  3  -  PROBLEM  ANALYSIS 

This  factor  evaluates  the  employee’s  potential  for  pro¬ 
motion  based  upon  how  well  the  employee  identifies 
problems  and  proposes  solutions.  Problem  analysis 
includes  such  areas  as  isolating  problems  by  major 
factors;  collecting  relevant  background  data;  and  pro¬ 
posing  solutions  which  consider  the  important  trade  offs; 
and  utilizing  innovative  approaches  when  necessary. 
<  Deciding  among  alternative  sola  hone  ia  nor  included  in 
this  lactor.) 

STEPS 


STEPS 

HIGH  POTENTIAL  -The  employee  organizes  and 
schedules  resources  such  that  new  or  unanticipat¬ 
ed  work  can  be  accommodated  with  minimum  impact 
7-  r—  on  other  v  oik  needs.  Goals,  schedules,  and  mile¬ 
stones  a  e  optimally  established  and  achieved 
within  tks  group.  Overall  work  administration  is 
.  such  as  r-i  maximize  subordinates’  productivity. 


6- 


HIGH  POTENTIAL  -  The  employee  quickly  and 
accurately  recognizes  which  problems  are  the 
most  complex  or  unprecedented,  and  identifies 
the  novel  or  unique  factors  which  must  be  Con¬ 
sidered  for  a  solution.  Is  alert  to  alternate  ap¬ 
proaches,  recognizes  novel  relationships  between 
[the  factors,  and  proposes  comprehensively  opti¬ 
mized  solutions.  Frequently  proposes  useful 
unique  or  innovative  approaches,  but  does  not 
propose  complex  solutions  to  simple  problems. 


$- 


4- 


AVERAG:  POTENTIAL  -  The  employee  ede- 

fw-rt.)  >Cn<»r..«sW>w  we-  tours  taeeam.ee.  we  pane  be.  nyilM 

Establishes  realistic  goals,  schedules,  and  prior¬ 
ities  for  subordinates.  Effectively  menagea  re¬ 
sources.  Recognizes  the  need  for  additional  re¬ 
sources  |  manpower,  rime,  dollars)  before  programs 
are  adversely  impacted. 


5- 


AVERAGE  POTENTIAL  -  The  employee  iden¬ 
tifies  problems  and  clarities  them  by  isolating 

-^*1  — <  -«  pr.U,  »*-*--g  \ 

quired  data  and  proposes  solutions  which  con¬ 
sider  most  of  the  important  trade  offs  with  little 
supervisory  assistance.  Will  occasionally  suggest 
unanticipated  or  innovative  approaches  to  solving 
problems. 


3- 


3- 


LOW  POTENTIAL  -  The  employee  adequately 
administers  planned  work  but  has  difficulty  with 
new  or  unanticipated  work.  Often  fails  to  estab¬ 
lish  realistic  goals  and  schedules  for  subordinates,  1- 

unnecaasarily  changes  individual  aasignmsnts, 
and  frequently  dans  joba  himnclf  rather  then  dele* 
gate  to  subordinates.  Does  nor  cltarly  establish 
priorities  for  the  group. 


LOW  POTENTIAL  -  The  employee  ia  able  to 
analyze  and  propose  acceptable  solutions  for 
routine  problems,  but  requires  frequent  assistance 
for  more  difficult  problems.  Shows  lrttln  creativity 
or  innovation. 


'Employees  designated  by  the  rater  and  by  the  reviewing  supervisor  ss  having  some  form  of  responsibility 
for  the  work  of  others. 


«»•»««  Ml 
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FACTOR  4  -  DECISION  MAKING 

(Supem*ers/Mjnaim  and  other  designated  employees**  j 

This  factor  evaluates  the  employee’s  potential  lot  pro¬ 
motion  based  upon  how  well  the  employee  evaluates  solu¬ 
tions  (own  or  others' )  and  makes  decisions.  Decision 
making  includes  such  areas  as  considering  pertinent 
decision  factors;  recognizing  significant  effects  of  pro- 
posed  solutions;  recognizing  when  problem  analysis  and 
solution  generation  has  been  adequate;  and  being  able  to 
make  decisions  in  a  timely  manner. 

STEPS 


FACTOR  5  -  SPEAKING  ABILITY 

This  factor  evaluates  the  employee's  potential  for  pro¬ 
motion  based  upon  ability  to  express  ideas  when  speak¬ 
ing,  Speaking  ability  includes  such  areas  as  being  con¬ 
cise.  clear  and  well  organized,  expressing  ideas  appro¬ 
priately  for  the  audience;  and  being  persuasive. 

STEPS 

HIGH  POTENTIAL  -  The  employee  gives  clear, 
concise,  thorough,  well  organized  and  smoothly 
flowing  briefings  in  a  confident  manner  that  is 
almost  always  appropriate  for  the  audience. 
Presents  an  an  of  confidence,  can  persuade  a 
critical  audience,  handles  interruptions  and 
^difficult  questions  well,  and  encourages  surfac- 
ling  of  key  issues.  Is  equally  effective  during 
|  informal  discussions  or  formal  briefings. 


7- 


H1GH  POTENTIAL  -  Whether  routine,  complex  or 
controversial,  the  employee  thoroughly  reviews 
various  possible  solutions  to  problems/ issues  and 
evaluates  the  tradeoffs  and  possible  impacts  of 
the  different  alternatives.  Recognizes  when 
unique  possibilities  may  have  been  neglected  and 
if  a  decision  should  be  delayed  for  further  prob¬ 
lem  analysis.  Consistently  selects  the  most 
appropriate  course  of  action  in  a  timely  manner. 


7- 


5- 


AVERAGE  POTENTIAL  -  For  most  problems  the 
emplovee  reviews  the  various  solutions  proposed, 
ievalu  ites  all  reasonable  trade  offs  and  possible 
mpacts  of  the  different  alternatives,  recognizee 
inadequacies  in  existing  problem  analysis  and 
solutions.  Makes  timely  and  adequate  decisions. 
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AVERAGE  POTENTIAL  -  The  employee's 
briefings  progress  smoothly,  are  usually  appro¬ 
priate  for  the  audience,  and  are  delivered  in  a 
confident  manner.  Usually  handles  interruptions 
and  difficult  questions  well. 


3- 


2- 


1- 


LOW  POTENTIAL  -  The  employee  evaluates 
possible  solutions  to  routine  problems,  consider¬ 
ing  relevant  alternatives  and  normally  makes 
acceptable  decisions  on  time.  In  complex  ot 
controversial  areas,  is  often  reluctant  to  stake  a 
decision  ot  makes  poor  decisions. 


LOW  POTENTIAL  -  The  employee  sometimes 
has  difficulty  expiessing  ideas  in  a  clear  and 
well-organized  manner,  in  conversation  or  in  in- 
'  formal  briefings.  Has  difficulty  answering 
straight-forward  questions  adequately  and  some¬ 
times  appears  ill-at-ease. 


"Employees  designated  by  the  rater  and  by  the  reviewing  supervisor  as  having  some  form  of  decision 
making  responsibility. 


FACTOR  6  -  WRITING  ABILITY 


FACTOR  7  -  WORKING  RELATIONSHIPS 


This  factor  evaluate*  the  employee's  potential  for  pro¬ 
motion  based  upon  ability  to  express  ideas  in  writing. 
Writing  ability  includes  such  areas  as  being  concise, 
clear  and  well  organized.  being  grammatically  correct; 
expressing  ideas  appropriately  for  the  reader;  and  being 
persuasive. 

STEPS 


3- 


1- 


ItlGH  POTENTIAL  -  The  employee’s  writing  is 
consistently  of  the  highest  quality.  Ideas  are 
always  expressed  clearly  and  concisely.  Recom¬ 
mendations.  positions  and  arguments  are  always 
convincing.  Final  drafts  are  extremely  polished, 
rarely  requiring  revision. 


[AVERAGE  POTENTIAL -The  employee’s  writing 
is  usually  rorrect  in  grammar,  spelling,  sentence 
structure  and  punctuation.  Ideas  are  usually  ex¬ 
pressed  clearly  because  of  good  word  choice  and 
logical  organization.  Final  drafts  seldom  need  to 
[be  rewritten. 
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[LOW  POTENTIAL  -  The  employee's  writing 
irequires  frequent  revision  due  to  sentence  struc- 
or  punctuation*  Ideas  are  often  not  clearly 
expressed  because  of  poor  word  choice  and 
organization. 


This  factor  evaluates  the  employee's  potential  for  pro¬ 
motion  based  upon  how  well  the  employee  works  with 
other  people,  both  individually  and  in  groups.  Working 
relationships  include  such  areas  as  working  with  super¬ 
visors,  peers,  subordinates,  support  and  staff  personnel, 
and  others  such  as  contractor  representatives;  keeping 
others  informed  of  work  status  and  changes  that  relate  to 
their  activities;  facilitating  and  stimulating  multi-number 
ot  interdisciplinary  activity;  and  behaving  in  a  profes¬ 
sional  manner. 

STEPS 


7- 


6- 


|  HIGH  POTENTIAL  -  The  employee  is  exception¬ 
ally  skilled  at  creating  harmonious  relationships 
with  others  and  among  members  of  diverse  groups, 
even  under  adverse  conditions.  Acts  as  an  inter¬ 
disciplinary' facilitator  or  coupler,  and  accelerates 
group  progress  by  clarifying  complex  problems  and 
resolving  deadlocks.  Always  maintains  a  superior 
information  flow  in  timely  fashion.  Always  uses 
knowledge  of  the  organizational  environment  and 
the  existing  informal  channels  in  effectively  work¬ 
ing  with  and  relating  to  others. 


(AVERAGE  POTENTIAL  -  The  employee  is  coop¬ 
erative  and  diplomatic  towards  others.  Assists  in 
group  progress  toward  its  goal  by  asking  pertinent 
questions,  responding  to  questions  by  others,  and 
contributing  to  the  resolution  of  issues.  Maintains 
a  good  information  flow,  utilizing  both  formal  and 
informal  organizational  channels. 


2- 


1- 


LOW  POTENTIAL  -  The  employee  is  usually 
ineffective  in  dealing  with  interpersonal  problems 
and  in  making  positive  contributions  to  group 
progress.  Does  not  always  maintain  nn  adequate 
flow  of  information  with  those  whose  work  is 
interrelated.  Does  not  always  present  a  favorable 
professional  image. 


FACTOR  8  -  WORK  LEADERSHIP 

Supervisors.  Managers  and  other  designated  employees* ) 

This  factor  evaluates  the  employee’s  potential  for  pro¬ 
motion  based  upon  how  well  the  employee  influences  and 
motivates  subordinates  or  other  personnel  to  work  effec¬ 
tively.  both  individually  and  in  teams.  Work  leadership 
includes  such  areas  as  monitoring  the  work  of  other 
employees;  providing  guidance  to  them;  giving  them 
understanding  of  overall  goals;  and  providing  opportunities 
for  participation. 


STEPS 


HIGH  POTENTIAL  -  The  employee  gains  a  very 
high  level  of  respect  and  confidence  from  subordi¬ 
nates  and  accepts  total  responsibility  for  their 
efforts.  Insures  that  everyone  understands  how 
their  work  relates  to  larger  organisational  goals 
and  max  m*zesthe  opportunity  for  all  to  contribute. 
Review:  subordinates'  work  and  deals  effectively 
with  si  ourdinates  whose  output  ranges  from  out¬ 
standing  to  very  marginal.  Stimulates  enthusiasm, 
creativi  y.  and  effective  self  management. 


AVESA  ZZ  POTENTIAL  -  The  employee  normally 
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and  accepts  responsibility  for  their  efforts.  Re¬ 
views  then  work  and  provides  meaningful  guidance. 
Guidance  is  often  sought  out.  Leads  routine  group 
efforts  well  and  non-routine  efforts  adequately. 


LOW  POTENTIAL  -  The  employee  has  little 
beneficial  influence  on  subordinates  and  some¬ 
times  avoids  responsibility  for  their  efforts. 
L- — (Usually  accepts  responsibility  for  routine  group 
teaks,  but  avoids  responsibility  for  complex  or 
controversial  tasks. 


FACTOR  9  -  PERSONNEL  MANAGEMENT  &  EEO 
COMMITMENT 

(Supervisors/. Managers*! 

This  factor  evaluates  the  employee's  potential  for  pro¬ 
motion  based  upon  how  well  the  employee  carries  out 
personnel  management  responsibilities.  Personnel  man¬ 
agement  includes  such  areas  as  recruiting  candidates; 
filling  positions  on  the  basis  of  merit;  encouraging  and 
assisting  Subordinates  to  set  and  pursue  then  career 
goals;  encouraging  and  assisting  employees’ training  and 
career  development;  giving  valid  performance  ratings  and 
counseling;  and  securing  recognition  for  deserving  sub¬ 
ordinates  or  taking  corrective  actions  for  inadequate 
performance.  EEO  and  Affirmative  Action  are  elements 
of  all  personnel  management  responsibilities. 

STEPS 


HIGH  POTENTIAL  -  The  employee  aggressively 
searches  for  outstanding  candidates  to  fill  open 
positions,  irom  both  inside  and  external  to  the 
organisation.  Actively  identifies  subordinates’ 
training  needs  and  encourages  subordinates’  pto- 
fessional  growth  and  career  progression.  Pro¬ 
vides  frequent  counseling,  guidance  and  perform¬ 
ance  feedback.  Either  motivates  a  poor  performer 
to  do  better  work  or  takes  appropriate  corrective 
action.  Aggressively  utilizes  all  available  forms 
of  recognition  to  reward  outstanding  performers. 
Actively  supports  organization's  Affirmative 
Action  Plan. 

lavtpar.F  Drvrtrvrrai  _  Th»  .4*. 

quntely  fills  open  positions  on  the  basis  of  the 
candidates’ merit.  Is  alert  to  training  opportunities 
for  subordinates  and  encourages  their  participa¬ 
tion.  Conducts  required  appraisals  and  provides 
]  J  adequate  subordinate  counseling.  Secures  recog¬ 
nition  for  really  outstanding  performers  and  takes  . 
appropriate  correction  action  for  noticeably  poor 
performers.  Actively  supports  the  requirements  of 
the  organization's  Affirmative  Action  Plan. 


5- 


4-r 


3- 


n_ 


LOW  POTENTIAL  -  Employee  superficially  eval¬ 
uates  qualifications  of  candidates  when  filling 
open  positions.  Attaches  low  priority  to  subordi¬ 
nates'  career  goals,  training,  growth  and  profes¬ 
sional  progression,  but  will  usually  agree  to  sub¬ 
ordinates’  self-initiated  training  plans.  Must  be 
pushed  by  supervisor  to  accomplish  required  ap¬ 
praisals  and  seldom  provides  adequate  counseling. 
Almost  never  seems  to  have  the  time  required  to 
secure  recognition  for  outstanding  performers. 
Passively  supports  organization’s  Affirmative 
Action  Plan. 


—Employees  designated  by  the  rater  and  by  tha  reviewing  supervisor  as  having  some  form  of  responsibility 
for  the  work  of  others. 
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FACTOR  10  -  PERFORMANCE  UNDER  PRESSURE 


FACTOR  11  -  WORK  OUTPUT 


This  factor  evaluate* the  employee's  potential  for  promo¬ 
tion  based  on  how  well  the  employee's  usual  performance 
is  maintained  under  pressure.  Sources  of  pressure  in¬ 
clude  time  and/or  resource  constraints;  changing  condi¬ 
tions;  high  level  visibility;  unclear  task  requirements; 
and  other  adverse  conditions. 

STEPS 


HIGH  POTENTIAL  -  The  employee  is  always 
flexible  enough  to  handle  all  high  pressure  situa¬ 
tions  and  accepts  them  as  part  of  the  job.  Con¬ 
siders  these  as  challenges  and  delivers  high 
quality  products/ services.  No  issue  is  so  formi¬ 
dable  that  it  degrades  performance.  Has  a  stabili¬ 
zing  influence  on  coworkers  during  periods  of 
stress. 


6- 


5- 


AVERAC-E  POTENT' AL  -  The  employee  accepts 
pressure  uat  t*  when  faced  with  rhea  and 
performr  well.  1  c  :.c*.ned  by  potential  high- 
level  revie-rs  of  work  roitpj-.  but  prepares  for  them 
and  delive.a  work  <j!  good  quality. 

3-1 


2-1 


l-l 


LOW  POTENTIAL  -  The  employee  loses  effec¬ 
tiveness  when  routine  is  interrupted  by  need  for 
changing  requirement  s/ emphasis,  or  when 
addressing  a  high  priority,  short  lead-time  prob¬ 
lem.  Requires  supervisory  bolstering/ assistance. 
Avoids  high  pressure  assignments  or  always  has 
reasons  why  he/ she  should  noc  be  assigned  them. 


This  factor  evaluates  the  employee’s  potential  for  promo¬ 
tion  based  upon  total  work  output.  Work  output  includes 
quality,  quantity,  and  timeliness.  It  also  includes  com¬ 
pliance  with  relevant  regulations,  directives,  policies, 
and  instructions  and  deadlines  from  supervisors.  Quality 
includes  the  extent  to  which  thj  work  demonstrates  use 
of  basic  technical  /  professional  knowledge  and  appro* 
priate  new  developments  m  the  employee's  field  and 
related  fields. 


STEPS 


6- 


HIGH  POTENTIAL  -  The  employee  constantly 
demonstrates  an  outstanding  ability  to  properly 
mix  the  interdependent  requirements  for  quality, 
q*  antity  and  available  time  to  produce  an  optimum 
e:  d  product  substantially  in  excess  of  expecta¬ 
tions.  Consistently  reflects  a  thorough  under- 
s.jnding  of  new  technologies  and  their  applica¬ 
tions  in  own  and  related  fields.  Employee’s 
technical  advice  and  assistance  is  repeatedly 
i  sought  by  others.  Work  always  reflects  knowledge 
|  of  regulations,  directives  and  policies  and  only 
|  occasionally  are  brief  instructions  from  super- 
required. 


! 


[average  POTENTIAL  -  The  employee  can  be 

Ideoended  ponn  In  rondure  a  rw.vnnnkle  nnnnlitv 


4- 


3- 


!  o;  acceptable  quality  work,  on  lime,  with  minimal 

i  supervision.  Employee’s  work  usually  reflects 

i-7-j  policies,  regulations,  organizational  operating 
|  instructions  and  supervisory  guidance.  Work 

demonstrates  an  acceptable  level  of  expertise  in 
.  !  own  area  of  specialization  and  a  working  knowl¬ 

edge  of  related  technologies. 


2- 


1- 


LOW  POTENTIAL  -  The  employee  requires  ex¬ 
cessive  time  to  complete  most  routine  tasks. 
Output  is  of  minimally  acceptable  quality.  Super¬ 
visor  must  frequently  remind  employee  of  organi¬ 
zational  policy  and  prior  instructions,  and  provide 
guidance  as  to  appropriate  quality/quantity/tiae 
trade  offs. 


let 


-aoc  a  or t -acts 


56 


APPENDIX  B 

MERIT  PROMOTION  APPRAISAL  SYSTEM 
RATING  FORM  B  (TECHNICAL) 
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MERIT  PROMOTION  APPRAISAL  SYSTEM 
FORM  B 


DATE  OF  APPRAISAL  j 

PERIOD  or 

SUPER  VISION 

FROM 

TO 

NAME /TITLE  OF  IMMEDIATE  SUPERVISOR 

name  /title  of  revierinq  supervisor 

INSTRUCTIONS  FOR  COMPLETING  SUPERVISORY  APPRAISAL 

The  Ait  Fotce  Merit  Promotion  Program  requires  the  immediate  or  first  level  supervisor  to  complete  »n  appraise!  on 
each  employee  supervised.  This  Supervisory  Appraisal  system  has  been  iesigned  to  help  the  supervisor  make  and 
record  the  required  merit  promotion  appraisal. 

Steps  in  coirpletm}  the  fans: 

1.  Supervisor  enters  the  required  data  on  the  top  of  this  page. 

2.  Supervisor  reviews  each  factor  and  determines  the  proper  rating  as  discussed  in  the  next  section. 

Circle  the  proper  step  in  the  box  on  page  »2.  Circle  N/A  if  a  rating  factor  is  not  applicable. 

3.  Have  the  appraisal  reviewed  by  the  next  higher  level  supervisor;  resolve  differences;  both  supervisors 
sign  and  drte  the  form  in  the  space  provided  on  page  f  2. 

4.  Show  and  discuss  the  apprsisal  with  the  employee,  have  the  employee  also  sign  on  page  tti. 

5.  Complete  the  card  punch  data  requited  as  noted  in  the  instructions  on  page  a  Z 

6.  Tear  off  pages  1  and  2  and  send  to  ASD/DPCE,  unless  given  othrr  instructions. 

Rating  Employee  Potential: 

Each  of  the  eleven  Merit  Promotion  Appraisal  Factors  consists  of  three  parts: 

1.  A  basic  factor  definition  found  in  the  first  Sentence  of  the  definition; 

2.  Subfactors  in  the  remainder  of  the  definition  which  represent  activities  related  to  the  basic  factor  defini¬ 
tion;  and 

3.  A  seven  steo  rating  scale. 

As  an  example,  consider  Factor  t  -  Self  Management.  This  factor  relates  to  how  well  the  employee  organizes, 
schedules,  and  accomplishes  own  work.  Self  Management  includes  such  subfactors  as  the  use  of  resources, 
setting  priorities,  coping  with  change,  assuming  responsibility,  and  amount  of  supervision  required.  Some 
assignments  may  not  allow  the  employee  to  display  performance  in  all  five  of  these  subfactors.  Some  work 
assignments  may  involve  perfoimenre  related  to  Self  Management,  but  not  specifically  covered  by  any  of  the 
subfactors.  The  supervisor  must  decide  which  aspects  of  the  employee's  performance  are  in  the  basic  factor 
definition  and  should  be  included  in  the  rating. 

The  supervisor’s  rating  of  the  employee's  potential  mutt  be  based  on  his  or  her  knowledge  of  the  employee’s 
current  performance.  In  accordance  with  CSC  teguletions,  the  Merit  Promotion  Appraisal  factors  have  been 
carefully  designed  to  focus  on  elements  of  the  employee’s  job  which  are  important  for  success  in  higher-level 
work. 

When  rating  (he  employee's  potential,  tht  supervisor  should  give  appropriate  consideration  to  the  complexity 
and  diversity  of  (he  work  required  at  the  employee’s  grade  level.  For  example,  proficiency  at  work  of  a  partic¬ 
ular  level  of  complexity  and  diversity  might  indicate  high  potential  for  a  G.S.  7  Engineer.  However,  this  same 
level  of  proficiency  would  indicate  low  potential  lot  the  job  of  a  G.S.  13  Engineer. 

The  task  of  the  supervisor,  in  using  the  Merit  Promotion  Apprsisal  F  actors,  is  to  rate  on  s  scale  of  seven  steps 
the  employee’s  potential  for  success  in  higher  level  work.  Scale  steps  One,  Four  and  Seven  provide  examples 
of  low,  average  and  high  levels  of  performance  for  some  of  the  subfactors.  Steps  Two,  Three,  Five  and  Six 
represent  intermediate  degteea  of  potential. 

Continuing  with  Factor  1  -  Self  Management,  as  tn  example: 

Step  1.  Low  Potential  -  The  employee  is  nble  to  organize  and  schedule  routine  work  adequately,  but  usually 
requires  supervisory  assistance  in  otgamzing  more  difficult  work  and  in  setting  priorities.  Supervisory 
intervention  is  required  to  insure  eithef  completion  of  the  effort  or  stopping  the  effort  before  the 
point  of  diminiihing  returns. 
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btep  4.  Average  Potential  —  The  employee  is  adapt* 
able  to  changing  conditions  and  unforeseen 
events  in  organizing  and  scheduling  routine 
work.  The  supervisor  occasional!',  assists  the 
employee  in  planning  for  more  difficult  work. 
The  employee  takes  considerable  initiative  for 
managing  own  final  work  througn  completion. 

Step  7.  High  Potential  -  The  employee’s  organization 
of  work  always  results  in  the  most  efficient  use 
of  time.  Is  vety  adaptable-  to  constantly  chang¬ 
ing  workloads  and  unpredictable  contingencies, 
adjusting  priorities  accordingly,  and  the  super¬ 
visor  is  never  required  to  revise  the  employee's 
plans.  Assumes  full  responsibility  for  ail 
aspects  of  assigned  tasks,  assuring  that  the 
output  meets  the  needs  of  the  requestor. 

One  of  two  approaches  can  be  taken  in  rating  an  em¬ 
ployee  on  a  given  factor.  One  approach  is  to  consider 
the  factor  as  a  whole  and  rate  the  employee's  performance 
relative  to  the  basic  factor  definition  based  on  your  over¬ 
all  impression.  Using  this  approach,  you  would  decide 
between  a  rating  at  Step  2  and  a  rating  at  Step  3  by  simply 
looking  at  Steps  1  and  4  and  deciding  whether  the  em¬ 
ployee  is  closer  to  being  just  above  Step  1  or  just  below 
Step  4. 

A  second  approach  is  to  rate  the. employee  on  each  of  the 
subfactors  which  apply  and  then  average  these  ratings. 
Let  us  consider  a  very  simplified  example  of  how  two 
emoloyees  might  be  rated  on  Seif  Management  using  this 
approach,  the  first  employee  is  teusun»oiy  Muaytaule 
to  changing  conditions  out  has  never  produced  %  finished 
product  without  major  supervisory  intervention.  The 
second  employee  is  reasonably  adaptable  to  changing 
conditions,  has  never  needed  a  push  to  get  the  job 
finished,  but  often  times  needs  assistance  in  deciding 
which  of  several  tasks  desetves  most  of  his  or  her 
attention.  The  two  employees  would  be  rated  as  follows: 

Employee  / 

Employee  Behavior  Scale  Step 

Reasonably  adaptable  to  changing  conditions  4 

Never  produced  a  finished  product  without  major 
supervisory  intervention  t 

Average  2  or  3 


Employee  ll 

Employee  Behavior  Scale  Step 

Reasonably  adaptable  to  changing  conditions  4 
Never  needed  a  push  to  get  the  job  finished  7 

Often  needed  assistance  in  deciding  which  of 
several  tasks  deserved  most  of  his  or  her 
attention  •  3 

Average  4  or  5 

As  noted  earlier,  the  performance  fated  may  include 
aspects  related  to  the  basic  {actor  definition,  but  not 
specifically  coveted  by  any  of  the  subfactors. 

Whichever  rating  method  is  used,  the  supervisor  must 
srrive  st  a  single  rating  fot  each  factor  which  represents 
his  ot  her  best  judgement  of  the  employee's  potential  for 
success  in  higher  level  work,  relative  tn  each  basic 
factor  definition.  Futthermote,  the  supervisor  must  be 
prepared  to  explain  to  the  employee,  using  specific 
examples  of  the  employee’s  job  performance,  why  the 
employee  was  rated  as  he  she  was. 


FACTOH  1  -  SELF  MANAGEMENT 

This  factor  evaluates  the  employee's 
potential  for  promotion  based  upon 
how  well  the  employee  organizes, 
schedules,  and  accomplishes  own  work 
Self  Management  includes  such  areas 
as  the  use  of  available  resources; 
setting  priorities  among  activities; 
coping  with  changing  conditions  and 
unforeseen  events;  and  assuming  re¬ 
sponsibility  for  the  completion  of 
one's  own  work. 

STEPS 


7- 


6- 


HIGH  POTENTIAL  -  Is  always  found 
working  on  the  highest  priority 
or  most  important  task.  When 
faced  with  a  delay,  for  whatever 
reason,  immediately  redirects 
attention  to  other  important 
work,  or  otherwise  uses  time  con¬ 
structively.  Requests  supervisory 
reassignment  to  higher  priority 

L- _ J  ...  t  situation. 

Assumes  full  responsibility  for 
all  aspects  of  assigned  tasks,  yet 
recognizes  when  supervisory  de¬ 
cisions  are  required.  Assures  that 
the  product  represents  thi  best 
possible  trade-off  between  quality 
and  timelinessl 


5— ■ 


4- 


3- 


AVERAGE  POTENTIAL  —  Sometimes  requires 
i supervisory  assistance  to  determine  if 
jhe  or  she  should  redirect  attention  to 
jother  work.  Assures  that  the  product 
[meets  both  the  quality  and  schedule  re¬ 
quirements. 


LOW  POTENTIAL  -  Must  be  frequently  re¬ 
minded  to  work  on  the  most  appropriate 
task.  Supervisory  intervention  is  re¬ 
quired  to  insure  either  completion  of 
the  effort  or  stopping  the  effort  be¬ 
fore  the  point  of  diminishing  returns. 
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FACTOR  3  -  PROBLEM  ANALYSIS 


FACTOR  2  -  WORK  ADMINISTRATION 
(Supervisor*  and  other  designated  employ¬ 
ees)* 

This  factor  evaluates  the  employes'* 
potential  for  promotion  baaed  upon 
how  well  the  employee  organise*  and 
schedule*  the  work  of  other**  Work 
Administration  include*  such  area* 
as  assigning  duties  and  establishing 
goal*  for  others;  following  progress 
and  changing  assignments  accordingly: 
and  assuming  responsibility  for  the 
work  of  others, 

STEPS 


This  factor  evaluates  the  employee'* 
potential  for  promotion  baaed  upon 
how  well  the  employee  identifies 
problems  and  proposes  solutions* 
approaches,  or  interpretations  to 
fit  the  situation.  Problem  Analysis 
includes  ium  areas  ss  identifying 
problems;  collecting  relevant  dots; 
and  proposir ;  solutions  which  con¬ 
sider  the  important  tradeoffs  or 
conflicting  requirements;  and  utilis¬ 
ing  innovative  approaches  when  neces¬ 
sary.  (Deciding  among  alternative 
solutions  is  not  included  in  this 
factor. ) 


HIGH  POTENTIAL  -  la  able  to  organ¬ 
ize  work  and  schedule  assignments 
for  maximum  accomplishment.  Takes 
into  account  the  critical  elements 
of  the  task  and  other  foreseeable 
difficulties  that  could  affect 
personnel  assignment*  or  project 
schedule.  Handles  unforeseen 
problems,  yet  recognizes  when  the 
supervisor  should  be  consulted* 


HIGH  POTENTIAL  -  Quickly  and 
accurati  1/  recognises  which  prob¬ 
lems  arc  most  important  and  identifies 
the  factors  which  must  be  considered  for 
a  solution.  Is  alert  to  alternate  ap¬ 
proaches,  recognizes  differing  rs- 
lstionah.ps  among  the  factors,  and  pro¬ 
poses  optimal  solutions.  Consistently 
proposes  useful  or  innovative  approaches, 
but  doet  not  propose  complex  solutions 
to  simple  problems. 


AVERAGE  POTENTIAL  -  Organizes  the 
work  and  schedule*  most  assienments 
for  maxiaiA  accomplishment,  but 
occasionally  requires  supervisory 
assistance  in  order  to  identify 
factors  that  could  interfere  with 
project  completion.  Consults 
supervisor  when  unforeseen  problems 
seise. 


AVERAGE  POTENTIAL  -  Identifies  problems, 
collects  the  required  data,  or  information 
and  proposes  solutions  which  consider  most 
of  the  important  trade-offs,  with  little 
supervisory  assistance.  Will  occasionally 
suggest  useful  or  Innovative  approves  to 
solving  problem*. 


LOW  POTENTIAL  -  Needs  close  super¬ 
vision  to  Insure  that  work  is 
acceptably  organized  and  scheduled. 
Often  does  not  seek  supervisory 
assistance  until  the  situation  It 
out  of  hand. 


LOW  POTENTIAL  -  Analyses  and  proposes 
acceptable  eolations  for  routine  prob¬ 
lems,  but  requires  frequsnt  assistance 
for  more  difficult  problems.  Shows  little 
creativity  or  innovation. 


"Employees  designated  by  the  rater  and  hy  the  reviewing  supervisor  as 
having  some  form  of  responsibility  for  the  work  of  others. 
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r»CTO*  4'-  DECISION  MAXING 
(Supervisor*  and  other  designated 


Ploy***)* 


This  factor  evaluates  the  employee's 
potential  for  promotion  based  upon  hoe 
veil  the  employee  evaluate*  solutions  (ovn 
or  others')  and  makes  decisions.  Decision 
•Asking  include*  such  areas  as  considering 
pertinent  factors;  recognizing  significant 
effects  of  proposed  solutions;  recognizing 
vnen  problem  analysis  and  solution  genera¬ 
tion  has  been  adequate;  and  making  decisions 
in  a  timely  manner. 


STEPS 


2-1 


[nIGH  POTENT! Ai  -  Thoroughly  reviews 
various  possible  solutions  to  problems. 

| evaluates  the  trade-offs  and  considers 
possible  impact  of  the  different  al- 
tematives.  Recognizes  when  alternative 
approaches  nay  have  been  neglected  and 
if  the  decision  should  be  delayed  for 
supervisory  review.  Consistently  makes 
timely  and  high  quality  decisions. 


AVERAGE  POTENTIAL  -  for  most  problems, 
the  employee  reviews  the  various  identi- 
Llfied  solutions,  and  evalustes  reasonable 
(trade- offs  and  the  impact  of  tne  differ¬ 
ent  al*-?— - , 

equate  decisions. 


WW  POTENTIAL  -  Evaluates  possible  solu- 
A"  ™tin»  actuation.,  consider* 
relevant  alternative,,  „q  ,c 

table  decisions.  In  non-routine  situs- 
tiora*  is  often  reluctant  to  mak#  a 
^inaly  decision  or  makes  poor  decision*. 


fACTOR  5  -  SPEAKING  ABILITY 

.  Thia  factor  evaluates  the  employee's 
potential  for  promotion  based  upon 
ability  to  communicate  events  and 
ideas  when  talking .  Speaking  ability 
includes  such  areas  as  being  concise* 
clear,  ana  well  organised* 

STEPS 


HIGH  POTENTIAL  -  Conveys  Ideas  by 
talking  in  a  clear  and  concise 
In-snner-  Describes  events  and  pro¬ 
cedures  well,  and  responds  to  qoes- 
tions  directly.  Is  highly  effective 
'during  meetings  and  informal  dis¬ 
cussions. 


2-! 


AVERAGE  POTENTIAL  -  The  employs,  us- 
vally  conveys  Ideas  dearly  and  answer, 
[question,  directly .  7 


1-1  |  LOW  POTENTIAL  -  Tha  employee  1*  often 
■  nerd  to  understand. 


•Employees  desifnated  by  the  rater  and  by  the  reviewing  supervisor  as 
having  son*  form  of  decision  making  responsibility. 
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! 

[ 


FACTOR  6  -  WIWNG  AilUTT 

fact cr  evaluat •*  the  *«ployee'» 
for  promotion  b—d  upon 
tc  express  ides*  IP  writing. 
r.itiM  Ability  include.  .ueh  area. 

ai  SSg  eone1”'  c^let* 

ansi  wall  organised. 


HIGH  POTENTIAL  - 

consistently  expressed  legibly. 

clearly  and  concisely- 


1  AVERAGE  POTENTIAL  -  Information 
la  usually  legible  and  ideas  are 
usually  expressed  clearly. 


tow  POTENTIAL  -  Employees  writ¬ 
ing  ray  be  Illegible  and  he/Uie 
U  often  uj table  to  express  !*«•» 


factor  7  -  vorrino  relationships 

in  groups.  eorRine  "  Witn  super- 

cVude  such  are  support  and 

rr^oUl,  and  other*  —  -  «— 

and  eoneraetor  personnsl. 

STEPS 

CISTpOTEJITIAI  -  M-»*  “ln^ 

7  Jh^oniou*  working  relation^ip*. 
yleven  under  adverse  condition*. 


ttvSl  OS  POTENTIAL  -  I*  usually  co- 
loper  live  and  diplometlc  toward* 

loth*  *3  in  th«  work  ••ttint* 


r^TwTENTIAL  -  reUs  to  geT 
with  «ny  people 
1  flict  in  group  set -logs- 

I  .  ....  nmtcnt  a  favorable  prof- 


FACTOR  8  -  WORK  LEADERSHIP 

(Supervisors  and  other  desipiated  employees)* 


FACTOR  9  -  PERSONNEL  AND  ECO  COMUIKENT 
(Supervisors  only) 


This  fee tor  evaluate*  the  employee’s 
potential  for  promotion  based  upon  hoe 
veil  the  employee  influences  sod  moti¬ 
vates  subordinates  or  other  personnel 
to  work  effectively,  both  individually 
and  in  leans.  Work  Leadership  includes 
suer,  areas  u  monitoring  the  work  of 
other  employees;  providing  guidance  to 
then;  giving  then  understanding  of  goals; 
and  providing  opportunities  for  participa¬ 
tion. 


STEPS 


HIGH  POTENTIAL  -  Cains  a  very  high 
level  if  respect  and  confidence 
from  subordinates  and  accept*  re¬ 
sponsibility  for  their  efforts. 
Insures  that  everyone  understands 
hov  tleir  work  relates  to  the  work 
of  ott  ers  in  the  group,  and  maxi¬ 
mizes  the  opportunity  for  all  to 
contra  outs.  Reviews  subordinates' 
work  t  nd  deals  effectively  with 
subordinates  who's  output  ranges 
from  outstanding  to  very  marginal . 
Stimul  ites  enthusiasm,  creativity, 
and  effective  self  management. 


AVERA1S  POTENTIAL  -  Normally  gains 
respect  end  confidence  from  subordin¬ 
ates  and  accepts  responsibility  for 
their  wwwi  ut.  until-  progress 


| — 

1-1 _ I  LOW  P 

^ influ 
I  avoid 


___  POTENTIAL  -  Has  little  beneficial 
influence  on  subordinates  and  sometimes 
avoids  responsibility  for  their  efforts. 


This  factor  evaluates  the  employee’s 
potential  fir  promotion  based  upon  how 
well  the  employee  carries  out  personnel 
management  responsibilities.  Personnel 
Management  includes  such  areas  as  filling 
positions  on  the  basis  of  merit;  encourag¬ 
ing  and  assisting  subordinates  to  set  and 
pursue  their  career  goals;  giving  realistic 
performance  ratings  and  counseling;  and 
initiating  actions  to  secure  recognition 
for  deserving  subordinates  or  taking 
corrective  actions  for  inadequate  per¬ 
formance.  EEO  and  Affirmative  Action 
are  elements  of  all  personnel  management 
responsibilities. 


STEPS 


HIGH  POTENTIAL  -  Actively  identifies 
outstanding  candidates  to  fill  open 
l  aaitions.  Actively  identifies  sub¬ 
ordinates'  training  needs  and  encourages 
r inordinate* '  professional  growth  and 
progression.  Provides  frequent 
counseling,  guidance  and  performance 
f»edback.  Either  motivates  a  poor  per- 
i  >:\aer  to  do  better  work  or  takes  appro¬ 
priate  corrective  action.  Aggressively 
utilizes  all  available  forma  of  recogni¬ 
tion  to  reward  outstanding  performers, 
•xtlvely  supports  and  participates  in 
organisation's  Affirmative  Action  Plan. 


A'TT-VZZ  rC.-in**  -  MwvqwMteiy  mis  open 
positions  on  the  basis  of  candidate's  merit. 
Is  alert  to  training  opportunities  for  aub- 
ordinates  and  encourages  their  participation. 

~  ^  Conducts  appraisals  as  required  and  provides 
adequate  subordinate  counseling.  Secures 
recognition  fer  really  outstanding  per¬ 
formers.  Actively  supports  the  rsquire- 

3-  ,  ments  of  the  organisation’s  Affirmative 
Action  Plan. 


*4 


LOW  POTENTIAL  -  Do*,  not  fully 
consider  til.  qualification.  of  .11 
candidat.*  .hen  fill  in,  op.n  posi¬ 
tion..  Attaches  low  priority  to 
•ubordinetw '  car.tr  goal.,  train¬ 
ing,  growth  and  professional  pro- 
gm.ion  but  will'  uaually  agr«a  to 
aubordinatas'  aalf  initl.tad  train¬ 
ing  Plana.  Kuat  b.  puahad  by  aupar- 
viaor  to  accoepliah  required  ap¬ 
peal  ta.  a  and  aliaoat  never  aawaa  to 
have  th»  tine  raquirad  to  aocur. 
ra cognition  for  outstanding  par- 
foraera.  Paaaiv.iy  aupporta  or- 
ganiaation’*  Affiraativ*  Action 
Plan. 


*Eaploy.«,  daslgnattd  by  tha  ratar  and  by  th«  ravi.ing  auptr—isor  at  having  loaa 
fore  of  ratponaibillty  for  tha  work  of  other*. 
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FACTO*  10  -  PERFORMANCE  a© 1*  PRESSURE 

Thi»  faetcr  evaluated  the  employee's 
potential  fcr  promotion  Cased  upon  how 
veil  tno  erp'-oyae's  u»u«l  performance  ia 
raintained  under  pressure.  Sourest  of 
pressure  Include  tine  and/or  reaouree 
constraints,  and  other  adverae  condi¬ 
tion*. 


FACTOR  11  -  MORE  OUTPUT 

jhis  fee tor  evaluate*  the  employ**'* 
potential  for  promotion  baaed  upon 
total  vo rk  output.  Work  Output  in¬ 
clude*  Quality,  Quantity,  and  time¬ 
line**.  It  *l*o  include*  compliance 
with  relevant  regulation*,  directive#, 
policies,  and  inatruction*  »nd  dead¬ 
lines  from  supervisor.  Quality  in¬ 
cludes  the  extent  to  uhieh  the  work 
demonstrates  use  of  basic  technical 
knowledge  and  ne*  development*  in 
the  employee's  field. 

STEPS 


HIGH  POTENTIAL  -  Constantly  demon¬ 
strates  an  outstanding  ability  to 
produce  an  optimum  end  product  sub¬ 
stantially  in  exeeaa  of  normal  ex¬ 
pectations.  dork  consistently  re- 
-leeta  a  thorough  understanding  or 
re.  technologia*  or  regulations  and 
-.heir  application  in  own  field, 
j  Employ**'*  technical  adviea  and 

*  .  _ _ - _ ...tmdlu  tauiht 
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MERIT  PROMOTION  APPRAISAL  SYSTEM 
RATING  FORM  C  (CLERICAL) 
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MERIT  PROMOTION  APPRAISAL  SYSTEM 

FORM  C 


NAN*  IL»0I  —  Ft  ft  —  Wi*tf  »  ImuU 


ORGANIZATION  f To 


AO»IT*ON  (T,U0-  -  Oe*tMf 


' 

0*TE  OF  RPPRAISA*.  ] 

1 _ r< 

WOO  OF  SUPERVISION 

FROM 

TO 

NAME/ TITLE  OF  IMMC  D* A T C  SUPERVISOR 

MARE/  TITLE  OF  REVIEWING  SUPERVISOR 

INSTRUCTIONS  PO R  COMPLETING  SUPERVISORY  APPRAISAL 
(RETf'RN  TO  ASD/DPCE) 


The  Air  Forue  Merit  Promotion  Program  requires  the  immediate  or  first  level  supervisor  to  complete  an  appraisal  on 
each  employee  supervised.  This  Supervisory  Appraisal  system  has  been  designed  to  help  the  supervisor  make  and 
record  the  required  merit  promotion  appraisal. 

Steps  is  cow  pitting  lit  form: 

L  Supervisor  enters  the  required  data  on  the  top  of  this  page. 

Z  Supervisor  reviews  each  factor  and  determines  the  proper  rating  as  discussed  in  the  next  section. 

Circle  the  proper  step  in  the  box  on  page  «Z  Circle  N/ A  if  a  rating  factor  is  not  applicable- 

3.  Have  the  appraisal  reviewed  by  the  next  higher  level  supervisor,  resolve  differences;  both  super¬ 
visors  sign  and  date  the  form  in  the  space  provided  on  page  »Z 

4.  Show  and  discuss  the  appraisal  with  the  employee,  have  the  employee  also  sign  on  page  *Z 

5.  If  th-.  employee  does  not  already  have  a  copy  of  ASD  Fo  rn  389,  please  give  them  one. 

6.  Complete  the  card  punch  data  requited  as  noted  in  the  instiuctions  on  page  aZ 

7.  Tear  off  pages  1  and  2  and  send  to  ASD/DPCE,  unless  given  other  instructions. 

Raring  Employ ««  Forewarns.* 

Each  of  the  eight  Merit  Promotion  Appraisal  Factors  consists  o(  three  puts 
L  A  basic  factor  definition  found  in  the  first  sentence  of  the  definition; 

Z  Subfactors  in  the  remainder  of  the  definition  which  represent  activities  related  to  the  basic  factor  definition; 

and 

1  A  seven  step  rating  scale, 

As  an  example,  consider  Factor  1  -  St If  Management  This  factor  relates  to  how  well  the  employee  organizes, 
schedules,  and  accomplishes  own  work.  Self  Management  includes  such  subfactors  as  setting  priorities,  msktng 
appropriate  decisions,  assuming  responsibility,  and  development  of  filing  systems.  Some  assignments  may  not 
allow  the  employee  to  display  performance  in  ail  four  of  these  subfactors.  Some  work  assignments  may  involve 
performance  related  to  Self  Management,  but  not  apecificallv  coveted  by  any  of  the  subfactors.  The  superviaor  must 
decide  which  aspects  of  the  employee's  performance  are  in  the  basic  factor  definition  and  should  be  included  in 
the  rating. 

The  supervisor's  rating  of  the  employee's  potential  must  be  baaed  on  his  or  her  knowledge  of  the  employee's 
entrant  performance.  In  accordance  with  CSC  regulations,  the  Merit  Promotion  Appraisal  Factors  have  been 
carefully  designed  to  focus  on  elements  of  tha  employee’s  job  which  are  important  fot  success  in  higher-level 
work. 

When  rating  the  employee's  potential,  the  supervisor  should  give  appropriate  consideration  to  the  complexity  and 
diversity  of  the  work  required  at  the  employee's  grade  level.  For  example,  proficiency  at  work  of  t  particular  level 
of  complexity  and  diversity  might  indcate  high  potential  for  a  G.S.  7  Engineer.  However,  thia  same  level  otpro- 
ficiency  would  indicate  low  potential  fot  the  job  of  a  G.S.  13  Engineer. 

The  task  of  the  supervisor,  in  using  the  Merit  Promotion  Appraisal  Factors,  is  to  rate  on  a  scale  of  anven  ataps 
tha  mnployee's  potential  for  success  in  Higher  level  work.  Scale  atepa  One,  Four  and  Sevan  provide  examples  of 
low,  average  and  high  levels  of  performance  for  some  of  tha  ublactors.  Steps  Two,  Three,  Five  and  Six  represent 
intermediate  degrees  of  potential. 


Continued  on  page  3 

A  SO  389  xiST  wwcvioui  toiriow  rsentOLcrc. 
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Continuing  with  Factor  \  —  Saif  Management,  as  an 
example 

Step  1.  Low  Potential  —  Must  be  frequently  reminded 
to  work  on  the  most  appiopnate  task.  Super¬ 
visory  intervention  is  required  to  insure  either 
completion  of  the  effort  or  stopping  the  effort 
before  the  point  of  diminishing  returns 

Step  4.  Average  Potential  -  Sometimes  requires  super¬ 
visory  assistance  to  determine  if  he  or  she 
should  redirect  attention  to  other  work.  Assures 
that  the  output  meets  both  the  quality  and 
schedule  requirements. 

Step  7.  High  Potential  —  Is  always  found  working  on 
the  highest  priontv  or  most  important  task.  When 
faced  with  a  delay,  for  whatever  reason,  imme¬ 
diately  redirects  attention  to  other  important 
work,  or  otherwise  uses  time  constructively . 
Assumes  full  responsibility  for  all  aspects  of 
assigned  tasks,  yet  recognizes  when  supervisory 
decisions  are  requited.  Assures  that  the  output 
represents  the  best  possible  tradeoff  between 
quality  and  timeliness. 

One  of  two  apj.  loaches  can  be  taken  in  rating  an  employee 
on  a  given  factor.  One  approach  is  to  consider  the  factor 
as  a  whole  and  rate  the  employee's  performance  relative 
to  the  basic  factor  definition  based  on  your  overall  im¬ 
pression.  Ufir  g  this  approach,  you  would  decide  bet  ween 
a  rating  at  Step  2  and  a  rating  at  St^>  3  by  simply  looking 
a:  Steps  1  ar  d  4  and  deciding  whether  the  employee  is 
closer  to  bein;*  just  above  Step  1  or  just  below  Step  4. 

A  second  approach  is  to  rate  the  onployee  on  each  of  the 
subfactors  which  aooiy  and  then  average  these  ratings. 
Let  us  consider  a  very  simplified  example  of  how  two 
employees  might  be  rated  on  Self  Management  using  this 
approach.  The  first  employee  is  usually  found  working 
on  the  highest  priority  work,  but  has  never  produced  • 
timsfteo  product  wicnout  major  supervisory  intervention. 
The  second  employee  has  never  needed  a  push  to  get  the 
job  finished,  but  often  needs  assistance  in  deciding 
which  of  several  usks  deserves  most  of  his  or  her 
attention.  The  two  employees  would  be  rated  as  follows 


Employ  te  f 

Employee  Behavior  Scale  Step 

Usually  found  working  on  the  highest 
priority  work  4 

Never  produced  a  finished  product  without 
major  supervisory  intervention  l 

Average  2  or  3 

Employee  ff 

Employee  Behavior  Scale  Step 

Never  needed  a  push  to  get  the  job 
finidicd  7 

Often  needed  assistance  in  deciding 
which  of  several  tsskt  deserved  most 
of  his  or  her  attention  3 

Average  5 


As  noted  eadier,  the  performance  rated  may  include  a* 
pacts  rriated  to  the  basic  factor  definition,  but  not 
^eafically  covered  by  any  of  the  sub  factors. 


Whichever  rating  method  is  used,  the  supervisor  must 
amve  at  a  single  rating  for  each  factor  which  represents 
his  or  her  best  judgement  of  the  employee’s  potential  for 
success  in  higher  level  worn,  relative  to  each  basic 
factor  definition.  Furthermore,  the  supervisor  must  be 
prepared  to  explain  to  the  employee,  using  specific 
examples  of  the  employee's  job  performance,  why  the 
employee  was  rated  as  he/ she  we* 

FACTOR  i  -  SELF  MANAGEMENT 


This  factor  evaluates  the  employee’s  potential  for 
promotion  based  upon  how  well  the  employee  organises, 
schedules,  and  accomplishes  own  work.  Sel f  M anagement 
includes  such  areas  as  setting  work  priorities;  making 
appropriate  decisions;  assuming  the  responsibility  and 
taking  the  initiative  for  completion  of  own  work;  and  the 
development  of  filing,  and  control  or  tracking  systems. 


STEPS 


1-1 


HIGH  POTENTIAL  —  Is  always  bund  working 
on  the  highest  priority  or  most  important  task. 
When  faced  with  a  delay,  for  whatever  reason, 
immediately  redirects  attention  to  other  important 
work,  or  otherwise  uses  time  constructively. 
Assumes  full  responsibility  for  all  aspects  of 
assigned  tasks,  yet  recognizes  when  supervisory 
decisions  are  required  Assures  that  the  output 
represents  the  best  possible  trade-off  between 
quality  and  timeliness. 


AVERAGE  POTENTIAL  —  Sometimes  requires 
supervisory  assistance  to  determine  if  he  or  she 
(  should  re-direct  attention  to  other  work.  Assures 
that  the  output  meets  both  the  quality  and 
|  schedul  e  requirements. 


LOW  POTENTIAL  —  Must  be  frequently  remind 
ed  to  work  on  the  most  appropriate  task*  Super* 
visoty  intervention  is  required  to  insure  either 
completion  of  the  effort  or  stopping  the  effort 
[before  the  point  of  diminishing  return* 


FACTOR  2  -  WORK  ADMINISTRATION 
<D+s*i noted  employees9) 

This  factor  evaluates  the  employee’s  potential  for  pro¬ 
motion  based  on  how  well  the  employee  provides  guidance 
to  other  personnel.  Work  Administration  includes  such 
areas  as  reviewing  the  work  of  other  employees  for 
conformance  to  applicable  regulations  and  operating 
procedures,  and  for  accuracy,  clarity,  grammar,  and 
spelling;  providing  technical  guidance  and  training  on 
internal  policies  at. a  procedures  as  required;  and  estab* 
tuning  and  changing  workload  priorities  to  accomodate 
changing  conditions  and  unforeseen  events. 

STEPS 


FACTOR  3  -  PERSONAL  ASSISTANCE  ACTIVITIES 
fFS-OJlfi  Secretaries  only) 

This  factor  evaluates  the  employee's  potential  for  pro¬ 
motion  based  upon  participation  in  the  management  of 
the  supervisor's  program  or  immediate  office  as  a  pet- 
sonal  assistant.  The  personal  assistant  role  involves 
applying  a  thorough  knowledge  of  the  supervisor* s  views 
and  policies  in  such  areas  as  composing  routine 
correspondence  and  reports  scheduling  and  preparing 
for  appointments  and  meetings;  acting  as  a  liason 
between  supervisor  and  those  reporting  to  him/her,  and 
between  supervisor  and  other  offices;  routing  correspon¬ 
dence  to  subordinate  offices  and  reviewing  correspon¬ 
dence  pepared  for  supervisor’s  signature 


HIGH  POTENTIAL  —  When  indoctrinating, 
training,  reviewing  or  introducing  change,  the 
employee  provides  explanations  and  examples  at 
an  appropriate  pace  to  insure  proper  implemen¬ 
tation.  Assures  new  or  unanticipated  wodc  is 
accomplished  Thoroughly  reviews  the  work  of 
other  perso/nd  and  takes  appropriate  action. 
Encourages  and  guides  other  employees’  career 
development  in  cooperation  with  supervisors 
responsible  for  these  employees. 


HIGH  POTENTIAL  -  Participates  in  the 
management  of  the  supervisor’s  program,  thereby 
relieving  the  supervisor  of  handling  time  con¬ 
suming  ,  non- tech  meal  tasks.  Is  alert  to  and 

•  •cognises  situations  requiring  immediate  atten¬ 
tion.  Because  of  extensive  knowledge  of  super* 
'i-ior’s  views  and  policies,  acts  as  supervisor’s 
representative  in  dealings  with  high  level  pet* 
sonnel.  Provides  information  and  guidance  to 

*  acting”  supervisors  and  other  personnel  in  the 
absence  of  the  supervisor. 


AVERAGE  POTENTIAL  -  Adequately  indoc* 
trinates  and  trains  employees  by  showing  them 
basic  source  material  and  answering  questions. 
[Informs  other  emolnvees  of  chances  in  policies 
and  procedures.  Insures  that  routine  work  is 
accomplished,  and  that  new  or  unanticipated  wotk 
is  completed  with  minimal  disruptions.  Reviews 
the  work  of  other  employees  and  usually  takes 
appropriate  action. 


-  -  Must  be  pushed  to  indue- 
other  employees  and  avoids 
questions.  Provides  minimal 
of  other  personnel. 


AVERAGE  POTENTIAL  -  Participates  in  the 
l  odgement  of  the  supervisor’s  immediate  office, 
b«U  lr«|Ulft*»  auiuc  <u  deal¬ 

ing  with  complex  issues.  Trices  account  of  the 
importance  of  the  subject  matter  in  dealing  with 
correspondence  and  requests  for  appointments. 
Prepares  routinecorrespondence  without  guidance. 
Trades  progress  and  takes  necessary  actions  to 
assure  that  due  date,  are  met,  including  periodic 
requi  r  entente 


LOW  POTENTIAL  -  Require*  supervisory 
guidance  when  dealing  with  all  but  routinu  issue*. 
Is  not  consistently  effective  in  carrying  out  pee 
tonal  assistant  ml.  because  of  marginal  knowl¬ 
edge  or  misapplication  of  the  supervisors  views 
md  policies. 


-Employees  designated  by  the  rater  and  by  the  reviewing  supervisor  aa  having  ear  ban  of  responsibility 
for  the  work  of  others. 


*to  soma  ms  *ao«  ate  ?  naact 
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FACTOR  4  -  INFORMATION  PROCESSING 
(Clerk  Sped elt III  omlyl 

This  factor  evaluates  die  employee's  potential  for  pro¬ 
motion  based  upon  how  well  the  employee  processes 
infonsauon  in  a  specialized  field.  The  woik  may  in¬ 
clude  collecting  and  compiling  information;  analyzing, 
inteipreting,  and  summarizing  information;  making  oral  or 
written  recommendations  in  light  of  pertinent  regulations 
and  policies;  taking  action  and  providing  input  for  formal 
reports  and  bnefings. 

STEPS 


7-1 


HIGH  POTENTIAL  -  Quickly  and  accurately 
identifies  the  factors  which  must  be  considered. 
Is  alert  to  alternate  approaches  and  recognizes 
novel  relationships  among  factors.  Frequently 
proposes  useful  or  innovative  approaches  in 
conaonince  with  ^plicable  regulations  and 
policie. 


6- 


FACTOR  5  -  COMMUNICATION 


This  factor  evaluates  the  employee’s  potential  for  pro¬ 
motion  based  upon  ability  to  express  ideas  when  peak¬ 
ing  or  wnting,  and  based  upon  how  well  inquiries  and 
sensiliveinfomation  are  handled.  Communication  ability 
includes  such  areas  as  conciseness,  cl arity,  organization 
and  grammatical  accuracy. 


STEPS 


7-. 


HIGH  POTENTIAL  -  Wnttsn  and  verbal  com¬ 
munication  a  are  conaiatently  of  the  highest 
quality.  Telephone  calla  ere  handled  court eoualy 
with  prompt  transfer  of  information  or  referral  to 
Sipioptiate  office.  Uaea  good  judgement  in 
responding  to  questions  which  involve  organism 
tional  policies,  individual  privacy,  and  other 
sensitive  information. 


6- 


2- 


1-1 


AVERAGE  POTENTIAL  -  Identifies  mid 
clarifies  factors,  collects  the  required  information 
and  makes  proposals  or  takes  action  with  normal 
sunervivorv  assistance.  Regularly  suggests  use- 

#»«{  ft*  IQfWW arm**  asms  Jf*!*1*- 


LOW  POTENTIAL  -  Analyzes  iafonsstion  end 
makes  adequate  pope  salt  or  takes  action  for 
routine  issues,  but  requires  considerable  assis¬ 
tance  for  more  difficult  areas.  Occasionally 
suggests  unanticipated  or  innovative  approaches 
lor  actions. 


5— ) 


2- 


AVERAGE  POTENTIAL  -  Written  communica¬ 
tions  ar*  usually  dsar.  concise  and  onmpletm 
Telephone  calla  are  handled  courteously.  Uses 
adequate  judgement  in  responding  to  questions 
which  involve  organizational  policies,  individual 
privacy  and  other  sensitive  information. 


LOW  POTENTIAL  -  Sometimes  had  difficulty 
Caressing  ideas  or  relayinginfoimation  is  a  dear 
and  well  organized  manner.  Telephone  calls  are 
sometimes  handled  impropedy.  such  as  leaving 
the  caller  on  hold  for  long  periods  of  time,  not 
taking  completa  information,  or  not  offering  to 
take  messages.  Occasionally  reqionds  inappro¬ 
priately  to  questions  involving  organizational 
policies,  individual  privacy,  or  other  sensitive 
information. 
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APPENDIX  D 

FOLLOW-UP  QUESTIONNAIRE,  STUDY  THIRTY, 
IMMEDIATE  SUPERVISOR  RATING  METHOD 
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DEPARTMENT  Of  THE  AIR  FORCE 

wmqwwtl  AftONAUT.CAl  SYSTCmS  DMVON  ;jW5C) 

WWa*T*ATTU*ON  Ml  rOKI  IASI,  OHIO  4»4M 

uni  10  ^ 

dpc  15JUNJ973 

mmcti  Study  of  Merit  Proaction  Appraisals 

w.  Study  Participants  (Supervisor) 

1.  The  individual  whose  name  appears  on  the  top  of  the  attached 
rating  sheets  has  been  randomly  selected  as  part  of  a  study  of  the 
Merit  Promotion  Appraisals. 

2.  The  purpose  of  the  study  is  to  examine  the  effects  that  the 
viewpoint  of  the  rater  has  on  Merit  Promotion  Appraisals.  For  this 
study,  please  rate  the  employee  based  on  his  or  her  job  activities 
only.  Me  are  asking  for  your  view  of  the  "precise"  ratings  based 
only  on  the  warding  of  the  scale  steps.  Do  not  assume,  when  you 
rate  the  employee,  that  you  would  have  to  feed  back  the  ratings,  or 
he  concerned  about  the  impact  of  the  ratings  on  the  employee's 
opportunity  for  promotion. 

3.  Me  would  like  you  .i  fill  out  the  questionnaire  marked  "Supervisor” 
and  diatribute  the  other  three  sets  of  questionnaires  which  are 
attached.  Give  the  questionnaire  marked  "Self”  to  the  named  eraloyee. 
Give  the  questionnaire  narked  "Alternate  Supervisor"  to  another 
supervisor  at  your  level  or  above  who  has  some  knowledge  of  tV. 
employee's  work  activity.  Give  the  questionnaire  marked  "Co-worker" 

to  another  employee  at  about  the  same  level  as  the  named  employee, 
who  has  an  opportunity  co  observe  the  named  employee's  work.  Hetain 
the  questionnaire  mark si  "Supervisor"  and  complete  it  yourself. 

4.  Me  would  like  you  *o  label  each  of  the  four  questionnaire?,  with 
the  same  six  (4)  digit  record  number  so  that  the  ratings  can  bi 
linked  together  after  we  receive  the  completed  questionnaires.  You 
may  use  your  birthday,  ^art  of  your  phone  number,  or  any  other  six  (6) 
digit  number.  Please  do  not  uae  the  date  you  received  this  letter 
or  any  other  repeating  number,  such  as  999999,  or  a  series 
like  l  2  3  «  5  «. 

8.  Enter  the  random  number  on  each  of  the  four  questionnaires  in  the 
apaoe  for  "record  number". 

t.  Mo  are  also  asking  you  to  record  on  the  attached  "Supervisor" 
questionnaire,  the  ratings  you  actually  reported  for  the  employee. 

If  you  did  not  keep  a  copy  of  the  ratings  you  reported,  please  call 
83S04  or  9S6S4  and  we  will  send  you  a  copy. 


7.  Do  not  use  the  name  of  the  subordinate  rated  ir  this  questionnaire, 
The  ratines  are  anonymous . 

0.  Your  cooperation  in  completing  and  promptly  returning  the  question* 
naire  is  appreciated  and  will  help  insure  the  quality  of  the  ayatea. 

9.  This  questionnaire  is  self-addressed  for  return.  Please  fold 
as  shown  on  the  back  page. 

10.  THIS  STUDY  IS  COMPLETELY  VOLUNTARY  AND  ANONYMOUS . 

11.  Please  call  Dr  Norton  or  Dr  Balloun  at  Ext.  *i3S04  or  55(54  if 
you  have  any  questions  about  coeipleting  the  quest  onnaire. 


4  Atch 

1.  “Supervisor"  questionnaire 

2.  "Alternate  Supervisor" 
questionnaire 

3.  "Self"  questionnaire 

4.  "Co-worker"  questionnaire 


lBXKER,  Asst  Chief 
Civilian  Personnel  Division 
DCS/Personnel 
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EMPLOYEE  TO  BE  RATED 


. . . .  — —  —  «  'Cut  or  tear  here— - — - 

Please  leave  blank  any  questions  which  are  not  applicable  or  which 
you  do  not  wish  to  answer. 


SUPERVISOR 


1.  Record  number  -  the  supervisor  will 
randomly  assign  any  six  digit  number  to 
identify  the  employee.  This  number  should 
already  be  written  in  to  the  right  when  you 
receive  the  questionnaire. 

Several  different  kinds  of  raters  will  rate 
this  employee  and  this  record  number  is 
needed  on  all  of  their  forms  in  order 
to  allow  linking  of  the  different  ratings  for 
this  employee. 

2.  The  employee  was  rated  on:  _  (10) 

1  -  Fora  A 

2  «  Fora  B 

3  a  Fora  C  . 


3.  Today's  date:  MONTH _  (li> 

.DAY _  (13) 

YEAR _  (15) 

4.  What  is  your  organizational  symbol? 

[Home  Office  for  Collocates) 

Fill  in  ASD  2-ltr  symbol  ASD/ _  (17) 

or  check  one  of  the  options  below:  __  __  (19) 


AMRL  _ 

AFHRl  _ 

FTD  _ 

AFSC/PMQ  _  ' 

Other 

The  following  Question  is  optional 

5.  What  is  the  sax  of  the  employee  to  be  rated?  (21) 

(IwFemalej  2«Male) 

6.  If  you  are  military,  what  is  your  rank?  (25) 

(NCO,  2LT,  1LT,  CAP,  MAJ,  LTC,  COL) 

7.  tf  you  are  civilian,  what  is  your  grade?  __  _  (28) 

8.  What  is  the  grade  of  the  employee  to  be 

_  (30) 


ASD/ENA  _ 

AFWAL  _ 

ASO/ENE  _ 

AFAL  _ 

ASD/ENF  _ 

AFFDL 

ASD/ENO  _ 

AFML  __ 

or  ENS 

AFAPL  _ 

STUDY  NO  3_  0_  (1) 

CARD  NO  _  (3) 

RECORD  NO _ (4) 


APPENDIX  E 

79  FOLLOW-UP  QUESTIONNAIRE,  STUDY 
THIRTY-ONE,  SELF  RATING  METHOD 
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DEPARTMENT  OP  THE  AIR  FORCE 

NUDOUAfTfU  AIIONAllDCAi  iTjTlMj  DtVISiOM  (4# SC) 
WttGHT-MTTtKSOH  Alt  POtCI  IASI.  OHIO  AS  AM 


■mr  to 

AITM  Ot>  DPC 

tutitcT.  Study  of  Merit  Pronotion  Appraisals 


15  JUN  1379 


to  Study  Participants  (Self-rating) 

1.  You  have  been  randomly  selected  as  part  of  a  study  of  the  Merit 
Promotion  Appraisals.  You  are  being  asked  to  provide  a  rating  of 
yourself  from  your  viewpoint  as  an  employee. 

2.  The  purpose  of  the  study  is  to  examine  the  effect  that  the 
viewpoint  of  the  rater  has  on  Merit  Promotion  Appraisals.  For  this 
study,  please  rate  yourself  based  on  your  job  activities  only.  Me 
are  asking  for  your  rating  based  only  on  the  wording  of  the  scale 
steps.  Do  not  assume,  when  doing  the  rating,  that  you  would  have 
to  feed  back  the  ratings,  or  be  concerned  about  the  impact  of  the 
ratings  on  your  opportunity  for  promotion. 

3.  The  rating  sheet  which  you  return  has  been  coded  by  your 

svft rviicr  ‘.1.  a-rinderC-y.  tilcit.i  r.umicr.  thiuli  t;ar  .r  cut 
your  name  off  the  top  of  the  rating  sheet. 

4.  Please  complete  the  appraisal  within  five  days  of  receipt.  The 
rating  sheet  is  self-addressed  for  return.  Please  fold  as  shown  on 
the  back  page. 

3.  Your  cooperation  in  completing  the  appraisal  ia  appreciatea  and 
will  help  insure  the  quality  of  the  Merit  Promotion  Appraisal  System. 

6.  THIS  STUDY  IS  COMPLETELY  VOLUNTARY  AND  ANONYMOUS. 

7.  Please  call  Dr  Ncrton  or  Dr  Balloun  at  Ext.  33504  or  55654  if 
you  have  any  questions. 

- 

ROBERT  L.  BAKER,  Asst  Chief 
Civilian  Personnel  Division 
DCS /Personnel 


1  Atch 

Questionnaire 
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EMPLOYEE  TO  BE  RATED 


- — - . gut  or  tear  here .  . . — . 

Please  leave  blank  any  questions  which  are  not  applicable  or  which 
you  do  not  wish  to  answer. 


SELF 


STUDY  NO  _ _  (1) 

CARD  NO  _  (3) 

1.  Record  number  -  the  supervisor  will  RECORD  NO  _ _ _ _  _ _  (A) 


randomly  assign  any  six  digit  number  to 
identify  you.  This  number  should  already 
be  written  in  to  the  right  when  you  receive 
the  questionnaire.  i 

Several  different  kinds  of  raters  will  rate 
you  and  this  record  number  is  needed  on  all 
of  their  forms  in  order  to  allow  linking  of  ' 
the  different  ratings  for  you. 

2.  You  were  rated  on:  __  (10) 


1  «  Fora  A 

2  m  Form  B 

3  >  Fora  C 


4  a.*.. 

MONTH  _ 

()»> 

DAY _ 

(13) 

YEAR _ 

(15) 

4.  What  ia  your  organizational  symbol? 

(Home  Office  for  Collocates) 

Fill  in  ASD  2-1 tr  symbol 

ASD/ _ 

(1?) 

or  check  one  of  the  options  below: 

(19! 

ASD/ENA 

AFWAL 

AXRL  __ 

ASD/ENE 

AFAL 

AFHRL 

ASD/E.NF  _ 

AFFDL  _ 

FTD  _ 

ASD/ENO 

AFML  _ 

AFSC/PMQ 

or  ENS 

AFAPL  _ 

Other  _ 

The  following  Question  is  optional 
5._What  is  your  sex?  (1  ■  Females  2  ■  Male) 


6.  What  is  your  grade? 


(25) 


6.  Please  record  your  "precise"  in'  the  “spaces "pToviBed'below, 


Form  A  or  D  Form  C 


1. 

Self  Management 

— 

(33) 

1. 

Self  Management 

_  (33) 

a. 

Work  Administration 

_ 

(36) 

2. 

Work  Administration 

—  <3&: 

3. 

Problem  Analysis 

■M 

(37) 

3. 

Personal  Assistance 

_  (37) 

4. 

Decision  Making 

— 

(39) 

4. 

Information  Processing 

_  (39) 

5. 

Speaking  Ability 

MM 

(41) 

S. 

Communication 

_  (41) 

6. 

Writing  Ability 

_ 

(43) 

6. 

Working  Relationships 

_  (43) 

7. 

Working  Relationships 

(45) 

7. 

Performance  under  Pressure 

_  (45  > 

a. 

Work  Leadership 

- 

(47) 

S. 

Work  Output 

_  (47) 

9. 

Personnel  Management 

(49) 

10. 

Performance  under  Pressure 

(51) 

' 

u. 

Work  Output 

•(53) 
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AO-Alll  374 
UNCLASSIFIED 


AIR  FORCE  INST  OF  TECH  WRIGHT-PATTERSON  AFB  OH  SCHOOL— ETC  F/G  5/9 
AERONAUTICAL  SYSTEMS  DIVISION'S  MERIT  PROMOTION  APPRAISAL  SYSTE— ETC(U) 
SEP  81  W  K  MALONE 
AFIT-LSSR-76-81 


NL 


DEPARTMENT  Of  THE  AIR  FORCE 
Mwwurm  MtONAuncM  mn mi  smson  i<iki 
WMWMHWOII  Alt  fOKI  UU  OHIO  AMU 


itny  to 
uim  oti  OPC 


!5  JUN1973 


tuMCti  Study  of  Merit  Proaotion  Appraisals 


to.  Study  Participants  (Co-worker) 

1.  The  individual  whose  name  appears  on  the  top  of  the  next  page  has 
been  randonly  selected  as  part  of  a  study  of  the  Merit  Proaotion 
Appraisals.  You  are  being  asked  to  provide  a  rating  of  this  individual 
froa  your  viewpoint  as  a  fellow  employee. 

2.  The  purpose  of  the  study  is  to  exaaine  the  effect  that  the 
viewpoint  of  the  rater  has  on  Merit  Proaotion  Appraisals.  For  this 
study,  please  rate  the  employee  based  on  his  or  her  job  activities 
only.  Me  are  asking  for  your  rating  based  only  on  the  wording  of 
the  scale  steps.  Do  not  assume,  when  you  rate  the  employee,  that 
you  would  have  to  feed  back  the  ratings,  or  be  concerned  about  the 
impact  of  the  ratings  on  the  employee's  opportuiity  for  proaotion. 

3.  The  rating  sheet  wh'.ch  you  return  has  beer,  ceded  by  the  employee's 
supervisor  with  a  randomly  selected  number.  You  should  tear  or  cut 
the  employee's  name  off  the  top  of  the  rating  sheet. 

4.  Please* complete  the  appraisal  within  five  days  of  receipt.  The 
rating  sheet  is  self-adc rested  for  return.  Please  fold  as  shorn 
an  the  back  page. 

5.  Your  cooperation  in  completing  the  appraisal  is  appreciated  and 
will  help  insure  the  qu  ility  of  the  Merit  Promotion  Appraisal  Syatea. 

6.  THIS  STUDY  IS  COMP'-'. TEL Y  VOLUNTARY  AND  ANONYMOUS. 


7.  Please  call  Dr  Norton  or  Dr  Balloon  at  Ext.  S3904  or  35654  if 
you  have  any  questions. 


Civilian  Personnel  Division 
DCS /Personnel 


1  Atch 

Questionnaire 
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Please  leave  blank  any  question*  which  are  not  applicable  or  which 
you  do  not  wish  to  answer. 

CO-WORKER  RATING 


STUDY  NO  J3  2_  (I) 

CARD  NO  ' _  (3) 

1.  Record  number  -  the  supervisor  will  RECORD  NO _  (4) 

randomly  assign  any  six  digit  number  to  ”  ' 

identify  the  employee.  This  number  should 
already  be  written  into  the  right  when 
you  receive  the  questionnaire. 


Several  different  kinds  of  raters  will  rata 
this  employee  and  this  record  number  la 
needed  on  all  of  their  forms  in  order  to  allow 
linking  of  the  different  ratings  for  thia 


employee. 

2.  The  employee  was  rated  on: 

UO) 

1  *  Fora  A 

2  «  Fora  B 

3  *  Fora  C. 

3.  Today's  date: 

nCMin  _ _ 

(U> 

DAY _ 

(13) 

YEAR _ 

(15) 

4.  What  is  your  organizational  symbol? 
(Home  Office  for  Collocates) 

FILL  in  ASD  2-1 tr 

symbol  ASP/  __ 

(17) 

or  check  one  of  the  options  below:. 

(19) 

ASD/ENA 
ASD/ENE 
ASD/SNF 
ASD/ENO 
or  ENS 


AFWAL  _ 

AMRL  _ 

AFAL 

AFHRL 

AFFDL 

rro 

AFML 

AFSC/PW) 

AFAPL 

Other  _ 

The  following  question  is  optional 


5.  What  ia  the  sex  of  the  employe*  to  be  rated?  _  (21) 

(1  ■  Female;  2  ■  Male) 


6.  If  you  are  military,  what  la  your  rank? 

(NCO,  2LT,  1LT,  CAP,  KAJ,  LTC,  COL)  _  _  _  (28) 

7.  if  you  are  civilian,  what  la  your  grade?  _  __  (28) 

8.  What  ia  the  grade  of  the  employee  to  be 

rated?  _ _  (SO) 
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8. 

1. 

2. 

3. 

4. 

5. 

e. 

a. 

9. 

10. 

a. 


Please  record  your  "precis*"  ratings  in  the  spaces  provided  below. 
FORM  *  OR  B  fglttt  C 


Self  Management 

_  (33) 

1. 

Self  Management 

__ 

(33) 

Work  Administration 

—  <35> 

2. 

Work  Administration 

(35) 

Problem  Analysis 

—  <37> 

3. 

Personal  Assistance 

_ 

(37) 

Decision  Making 

_  (39) 

4. 

% 

Information  Processing 

esamm 

(39) 

Speaking  Ability 

_  (41) 

5. 

Communication 

— 

(41) 

writing  Ability 

—  («) 

6. 

.Working  Relationship* 

(43) 

working  Relationships 

_  (45) 

e* 

0  e 

Performance  under  Pressure 

(45) 

Work  Leadership 

_  (47) 

a. 

Work  Output 

(47) 

Personnel  Management 

_  (49) 

Performance  under 
Pressure 

—  <»> 

Work  Output 
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DEPARTMENT  OF  THE  AIR  FORCE 
KiMunui  AiiOMAunui  miiw  bmuon  imk) 
WUOHt'MmUOM  Ml  KMO  Ml  OHIO  4I4SS 


Mim  Study  of  Merit  Promotion  Apprainli 


•at  Study  Participant*  (Alternate  Superviaor) 

1.  The  individual  whose  name  appear*  on  the  top  of  the  next  page 
ha*  been  randomly  selected  as  part  of  a  study  of  the  Merit  Promotion 
Appraisal*.  Although  you  do  not  supervise  this  employee,  you  are 
being  asked  to  provide  a  rating  from  your  viewpoint  as  a  supervisor 
who  is  familiar  with  his  or  her  work. 

2.  The  purpose  of  the  study  is  to  examine  the  effect  that  the 
viewpoint  of  the  rater  has  on  Merit  Promotion  Appraisals.  For  this 
study,  please  rate  the  employee  based  on  his  or  her  job  activities 
only.  We  are  asking  for  your  rating  based  only  on  the  wording  of 
the  scele  steps.  Do  not  assume,  when  you  rate  the  employee,  that  you 
would  have  to  feed  back  the  ratings,  or  be  concerned  about  the  impact 
of  th*  rating*  An  the  e— plAvsa'a  nvvii»hw<ty  for  pmMifinn, 

3.  The  rating  sheet  which  you  return  has  been  coded  by  the  employee's 
supervisor  with  a  randomly  selected  number.  You  should  tear  or 

cut  the  employee's  name  off  the  top  of  the  rating  sheet. 

4.  Please  complete  the  appraisal  within  five  days  of  receipt. 

The  rating  sheet  is  sell -addressed  for  return.  Please  fold  as  shown 
on  the  back  page. 

5.  Your  cooperation  in  completing  the  appraisal  is  appreciated  and 
will  help  insure  the  quality  of  the  Merit  Promotion  Appraisal  System. 

g.  THIS  STUDY  IS  CONPL'.TELY  VOLUNTARY  AND  ANONYMOUS. 

7.  Please  call  Dr  Norton  or  Dr  Balloun  at  Ext.  53304  or  3S6S4  if 
you  have  any  questions. 


Civilian  Personnel  Division  Questionnaire 

DCS /Personnel 
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EMPLOYEES  TO  BE  BATES 


■  . . — —  Cut  or  tear  here"  ■  — 

Please  leave  blank  any  questions  which  are  not  applicable  or  which  you 
do  not  wish  to  answer. 


ALTERNATE  SUPERVISOR 


STUDY  NO  _i  JL.  (1) 

CARD  NO  _  (3) 

1.  Record  number  -  the  supervisor  will  RECORD  NO  _  (4) 


randomly  assign  any  six  digit  number  to 
identify  the  employee.  This  number  anould 
already  be  written  in  to  the  right  then 
you  receive  the  questionnaire. 

Several  different  kinds  of  raters  will 
rate  this  employee  and  this  record  i. umber 
is  needed  on  all  of  their  forms  in  order 
to  allow  linking  of  the  different  forms 
for  this  employee. 


2.  The  employee  was  rated  out 

1  -  Form  A 

2  ■  Form  B 

3  *  Form  C 

3.  Today's  date: 

MONTH  _ 

(11) 

.  DAY _ 

(13) 

4.  What  is  your  organizational  symbol? 

(Home  office  for  collocates) 

YEAR  _ _ 

(15) 

Fill  in  ASD  2-1 tr  symbol 

ASD/ _ 

(1?) 

or  cheek  one  of  the  options  belowi 


ASD/ENA 
ASD/ENt 
ASD/ENP 
ASD/ENO 
or  ENS 


The  followlns  auestlon  is  ootional 

5. 

What  is  the  sex  of  the  eaployee  to  be  rated? - 

(1  ■  Female;  2  »  Male) 

—  — . . 

.  (21) 

6. 

If  you  are  military,  what  is  your  rank? 

(NCO,  2LT,  1LT,  CAP,  L7C,  COL) 

— 

(25) 

7. 

Zf  you  are  civilian,  what  is  your  grade? 

—  — 

(28) 

8. 

What  is  the  grade  of  the  eaployee  to  be 
rated? 

(80) 

AFVAL 

AFAL 

AFTDL 

Am 

AFAPl 


AMRL  __ 

AFHRL  _ 

rro  _ 
ATSC/PNQ 
Other  _ 


(15) 
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8. 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 


Please  record  your  "precise"  ratings  in  the  spaces  provided  below. 
Form  A  or  B  Pons  C 


Self  Management 

_  (33) 

1. 

Self  Management 

_  (33) 

Work  Administration 

_  (35) 

2. 

Work  Administration 

_  (35) 

Problem  Analysis 

_  (37) 

3. 

Personal  Assistance 

_  (37) 

Decision  Making 

_  (39) 

4. 

Information  Processing 

__  (39) 

Speaking  Ability 

_  (41) 

5. 

Communication 

Writing  Ability 

_  (43) 

6. 

working  Relationships 

__  (43) 

Working  Relationships 

_  (45) 

7. 

Performance  under  Pressure 

__  (45; 

Work  Leadership 

_  (47) 

8. 

Work  Output 

_  <47‘ 

Personnel  Management 

_  (49) 

Performance  under  Pressure  (51) 

Work  Output 

_  (S3) 
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111  n 


Layout  of  Data  For  Validity  Analysis 


1-2  Study  Number 

3  Card  Number 

4-9  Record  Number 

10  Form 

11-12  Month 

13-14  Day 

15-16  Year 

17-18  ASD  Two-letter  Symbols 

19-20  Other  Symbols 

21  Sex 

22-24  Blank 

25-27  Military  Rank  if  Military 

28-29  Civilian  Grade 

30-31  Grade  of  Target  Employee 

32  Blank 

33  Study  Rating  for  Factor  1 

34  Actual  Rating  for  Factor  1  or  Blank 

35  Study  Rating  for  Factor  2 

36  Actual  Rating  for  Factor  2  or  Blank 

37  Study  Rating  for  Factor  3 

38  Actual  Rating  for  Factor  3  or  Blank 

39  Study  Rating  for  Factor  4 

40  Actual  Rating  for  Factor  4  or  Blank 

41  Study  Rating  for  Factor  5 

42  Actual  Rating  for  Factor  5  or  Blank 

43  Study  Rating  for  Factor  6 

44  Actual  Rating  for  Factor  6  or  Blank 

45  Study  Rating  for  Factor  7 

46  Actual  Rating  for  Factor  7  or  Blank 

47  Study  Rating  for  Factor  8 

48  Actual  Rating  for  Factor  8  or  Blank 

49  Study  Rating  for  Factor  9 

50  Actual  Rating  for  Factor  9  or  Blank 

51  Study  Rating  for  Factor  10 

52  Actual  Rating  for  Factor  10  or  Blank 

53  Study  Rating  for  Factor  11 

54  Actual  Rating  for  Factor  11  or  Blank 


Note:  For  target  employees  rated  on  Form  C,  only  the  first 
eight  factors  are  filled  out.  The  actual  rating  data  is  available  on 
the  Study  Thirty  data  cards  only.  The  actual  rating  data  columns  are 
left  blank  on  the  other  study  cards. 
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301CG077930705790I  1  CAP  04  7666667576757577 

311300779 

321300779307057901  1  1204  77777777 

341300779307057901  1  LTC  04  6  7  5  7  3  4  6  7 

3013008401071379AC  2  LTC  12  67  6666736674  7777 

3113006401071379AC  2  12  5  554554  56 

3213008401071979AC  2  1112  54335654  45 

3413008401071179AC  2  1412  65554565  45 

3013055253071279  061  0905  7576  7675647575 

3113055253071079  061  05  77777777 

3213055253071079  061  0505  46  65454 

3413055253071279  061  LTC  05  5  6  65555 

3013122072070979  102  LTC  14  66  66  557755  6677 

311312207  070979  102  676464  64 

3213122072070979  102  1414  6  75765  66 

3413122072070979  142  1514  7  7  7  7  6  6  6 

3013163683072079  021  LTC  05 

311316368 

321316368  072079EN  1  0505  77777777777 

3413163683072079  021  1505  7  676767 

3013397303072379  031  1504  67  7777776777 

311339730 
321339730 

3413397303071979  031  14  7  66766 

3013415362071279AC  2  LTC  12  7777777655767776767777 

3113415361072079  142  12  77775676777 

3213415361071279YZ  2  CAP  12  77774677777 

341341536  072779AC  2  1412  77777777777 

3013715293070679  091  14  4  7766  66667677 

311371529 

3213715293072579  091  0504  5  6  6767 

341371529 

301371 81210709 79AE  1  MAJ  09  77  66  777766  7777 

311371812 

3213718121070979AE  1  1209  77767666767 

3413718121070979AE  1  LTC  0976767676767 

3013840301070679  012  1413  6676656666667766766666 

3113840301071279  012  13  66676677777 

321384030  071179  012  1313  6  7  6  6  6  6  7  6  7  6  6 

3413840301071079  012  1313  7  7  6  7  5  6  7  7  7  7  6 

3014032383071079  041  1405  666666  66777766 

3114032383071079  041  05  66767767 

3214032383071079  041  0505  66656666 

341403238 

3014035441071779  022  1412  6554556555665555555467 

3114035441071779  022  12  6  65567  56 

3214035441071379EN  2  1412  65655676576 

3414035441071379EN  2  1412  65655676576 

3014267071 

3114267071071379EN  2  11 


i 


321426707 

341426707 

3014279321071079  072 
3114279321070979  072 
3214279321070979  072 
3414279321070979  072 
3014321682 
311432168 
321432168 

3414321682071179  062 

3014561231 

311456123 

321456 1231070979AC  1 

341456123  070979AC  1 
3014730343070579  1 

3114730343070579  061 

3214730343080579  061 

341473034 
3014930793 
311493079 
321493079 

3414930793071679  141 
3015231741070979  092 
3115131741070979  092 
3215131741071179  092 
341523174 

3015296 543072079AE  1 
3115296543071 279AE  1 
3215296543071079AE  1 
3415296543072379AE  1 
3015434643071079SD  1 

3115434643071079SD  1 

321543464 

3415434643071079SD  1 
3015528301  SP  2 
3115528301071379SP  2 
3215528301071679SP  2 

3415528301071379SP  2 
3015528471 
311552847 

3215528471070679EN  2 
3415528471070679EN  2 
3015534331 

3115534331071779EN  2 
3215534331071179  032 
341553433  071279EN  2 
3015539 751071379EN 
311553975 

3215539751071179  032 
341 55397510711 79EN  2 


1412  5  6  556  56 

12  54665576  65 

1212  56566645545 
1312  65765665  56 


1409  5444  455  56 


0906  4  44354  54 

12  44444444444 

03  66  55666677 

03  6  5  6  5  7  7  7  7 

0403  7  7  7  7  7  7  7  7 


0703  4  4  6  5  4 

LTC  14  7777676777676766777767 
14  66676766766 

1414  76666657766 

07  777676  75657677 

07  777  7677 

0607  7  5  6557 

COL  07  656  4364 

MAJ  06  77  7777777777 

06  7  7  7  7  7  7 

1206  7  77777 

MAJ  09  77  665565556666  6677 

09  7  7  7  7  6  7  6  7  7 

0909  54444335445 
1109  65555555555 


1212  67776576663 
1412  66556676566 

12  76776667677 

1312  66766566677 

1412  77666676  66 

7777776777777777777777 

1313  65756566677 

1413  66776577677 


94 


L  I 

;  .  t 

( 
r 
I 


I 

I 


S 

1 

.1 


3015546683 
3115546683070979 
32155466 8 
341554668 

071 

04 

6  5 

4  5  4  4  5  6 

3015555551070679 

092 

1514 

6677777777777777777777 

311555555 

321555555 

341555555 

3015557303 

1 

1405 

656676  65777676 

3115557303071179EN  1 

05 

6  5 

5  5  5  7  5  6 

3215557303071179 

031 

0405 

4  4 

4  3  4  4  5  4 

3415557303072579 

031 

1505 

6 

6  7  6  6 

3015562951070579EN  2 

1311 

55 

4444334455 

4466 

3115562951070979 

022 

11 

6 

5  4  4  5 

5  6 

3215562951070579 

022 

0911 

7 

5  4  5  6 

5  6 

3415562951070579 

022 

1211 

6 

4  5  4  7 

4  6 

3015566101070579 

072 

MAJ  13 

6677667777667777667777 

311556610 

3215566101070679 

072 

1413 

6  7 

6  5  6  6  5  5  6 

6  6 

3415566101070679 

072 

1413 

7  6 

6  7  4  5  6  6  6 

5  6 

3015618813070579 

061 

1205 

7 

7  6  7  7  7 

3115618813070579 

061 

05 

7 

6  6  7  7  7 

3215618813070679 

341561881 

061 

0505 

7 

7  7  7  7  7 

3015860251072379 

032 

1514 

6666776677776666776677 

311586025 

3215860251071879 

032 

1314 

7  7 

7  7  7  7  7  7  7 

7  7 

3415860251072579 

032 

COL  14 

6  6 

5  6  5  6  7  6  5 

6  6 

3016147291071979YP  1 

MAJ  11 

65 

5555666677 

6655 

311614729  071379AC  1 

11 

6  6 

6  6  7  7  7  6  5 

5  6 

321614729  071379AC  1 

1411 

5  6 

4  4  7  6  6  5  5 

4  4 

341614729  071379AC  1 

1411 

5  6 

4  4  7  6  6  5  5 

4  4 

30161744  3070979DP  1 

1306 

77 

7766667777 

31162744 

32162744 

34161744 
3016319471 
311631947  071079 

021 

05 

321631947 

341631947 

3016491711070679 

311649171 

321649171 

341649171 

3017331273 

092 

1413 

54 

5454555477 

7666 

3117331273071179YX 

04 

7  7 

7  6  7  6  7 

32173312730711 79YX  1 

CAP  04 

6  6 

6  7  6  6  7  7 

341733127 

3017342413070979 

141 

LTC  05 

55 

55  44665555 

3117342413070979 

141 

05 

6 

5  6  6  6  6 

95 


3217342413070979  141 

0505 

4  2  4  4  3  4 

3417342413071179  141 

LTC  05 

6  5  7  7  6  6 

3017351122071179  062 

1414 

5555444460665555 

5555 

3117351122070579  062 

14 

655677755 

6  6 

3217351121071179  062 
341735112 

1314 

443455444 

4  3 

3017419293070579PM  1 

1506 

666676  66777676 

311741929  070579PM  1 

06 

6  6  6  7  7  7  7 

3217419293070579PM  1 

0606 

6  6  7  6  6  6  7 

3417419193070579PM  2 

MAJ  06 

7  7  6  6  7  7  7 

3017771213071679YX  01 
311777121 

321777121 

341777121 

MAJ  04 

7  7  7  7  7 

3017854891072379  032 
311785489 

1514 

6555766666666655 

5566 

3217854891072079  032 
34  785489 

1414 

35555565 

4  4 

3018137993080879AE  1 

311813799 

1505 

67  66  66776667 

3218137993080979AE  1 

COL  05 

6  6  6  7  6  7 

34181379930808 79AE  1 

1505 

67  66  66776667 

3018210003072579  041 
311821000 

321821000 

341821000 

1505 

677777  77777777 

3018336751062679  092 

1514 

77  66  555577 

6666 

3118336751072679  092 

14 

665544556 

6  6 

3218336751072679  092 

CAP  24 

766667766 

7  7 

3418336751072679  092 

1514 

65754565465 

30186 46203070679YP  1 

1305 

6  6  5  5  6  7 

3118646203070679YP  1 
321864620 

341864620 

05 

65556666 

3019131362072079  092 

1305 

77  6666665566 

6767 

3119131362071179  092 

05 

777776777 

7  7 

3219131362072079  092 

0805 

6  6  6  5  7  6 

6  6 

3419131362070679  051 
3019249251 

13  5 

66665665 

5  6 

311924925  071579EN  2 

14 

776666766 

6  7 

3219249251071379EN  2 

1314 

676677776 

6  7 

3419249251071379EN  2 

1314 

676677776 

6  7 

3019812763071079AC  1 

311981276 

CAP  03 

66  6677665566 

3219812763071079AC  1 

0503 

70076777 

341981276  070679AC  1 

1103 

7  7  6  7  6  7 

3019886543072079PM  1 

1405 

455546  56564545 

3119886543073079PM  1 

05 

66666677 

3219886543072079PM  1 
341988654 

1205 

55546655 

96 


3012234563 

3112234563071779 

062 

06 

7  7  6  6  7  7 

3212234563071779 

341223456 

062 

0606 

6  6  5  5  6  6 

3012239563071779 

062 

NCO  06 

77  7766666677 

3112239563071779 

062 

06 

7  7  6  6  7  7 

3212239563071779 

341223956 

062 

0606 

6  6  5  5  6  6 

3012427251070979 

032 

1512 

5544555533445555 

5555 

3112427251070979 

032 

12 

54544355 

4  4 

3212427251070979 

032 

1212 

55764455 

7  6 

3412427251070979 

032 

1312 

54653456 

6  6 

3012504401072579 

012 

MAJ  12 

56  5656665666 

6656 

311250440 

3212504401071379 

341250440 

012 

1213 

6  6  6  5  5  4 

5  6 

3012539511071979 

032 

1412 

6677776676667777667777 

311253951 

3212539511072379 

032 

1512 

767766766 

6  7 

341253951 
3012332331 
311233238 
321233238  071079 

021 

05 

341233238 

3012543762071079 

052 

1312 

77  676655546767 

6677 

3112543762070979 

072 

12 

65674374 

5  6 

3212543762070979 

341254376 

072 

1212 

7  7  6  4  4  5  6 

7  7 

3012624631072079 

061 

1409 

7  7  6  7  7 

7  7 

311262463  072079 

062 

09 

5  4  4  6  4  6 

5  5 

3212624631072079 

061 

1209 

5  6  5  5  6 

5  6 

3412624631072079 

3012641321 

062 

1509 

7  6  5  6  7 

6  7 

311264132  0706 79SD  2 

09 

4545445  3 

5  4 

321264132 

3412641321070979SD  2 

1409 

55542641 

5  5 

3012624671 

311262467 

321262457 

3412624671072079 

062 

1509 

7  6  5  6  7 

6  7 

3012754101071379 

071 

1313  6  55556 

5  6 

3112754101071679 

071 

13 

555556756 

5  5 

3212754101071879 

071 

1213 

54454454433 

341275410 

30129432330706 79XR  1 

LTC  06 

3112943233071079AC  1 

06 

57777757 

3212943233070979XP  1 

1106 

4  4  2  3  4  2  3 

3412943233071 179AC  1 

MAJ  06 

43443233 

3010405381070679  072 

3110405381070679  072 

3210405381070679  012 

3410405381071079  072 

3010543363070979PM  1 

311054336 
321054336 

341054336307 1079SD  1 

3010603471071979  022 

3110603471071079PM  2 

3210603471071679  022 

3410603471071679  022 

3010629321 

3110629321071879  042 

3210629322071779  042 

341062932 

3010730541070979YZ  2 

311073054 

3210730541071679YZ  2 

341073054  070679  032 

3010910661071279  072 

311091066 

3210910661071679  072 
341091066 

3011107423071779  071 

3111107423071779  071 

3211107423071379  071 

3411107423071379  071 

301 1 il 1411070979EN  2 

3111111411071379  2 

3211111411070979  032 

341111141 

3011201451070679  072 

311120145 

3211201452070979  072 

3411201451070979  072 

3011229321071379  012 

311122932 

3211229321071379  012 

3411219211071379  042 

3011233211070579DE  5 

3111233211070579DE  2 

321 1 23321 1070579DE  2 
3411233211070579DE  2 

3011234561 
311123456 
321123456 

341123456 1072679SD  2 

3011290121 

311129012 


072 

1412  7667777743667766 

5465 

072 

12  5  554665 

5  5 

012 

1212  7  555565 

6  6 

072 

1312  6  56466 

6  5 

05  55 


66665566 


MAJ  05  5  6  7  6  6 

1413  7766666644556677776677 

13  76775577667 
1313  66655556545 
1313  66655556545 

14  66676676666 

1414  66555655  56 

1513  7666777666767777767777 

LTC  13  76777677667 
1513  76777767  77 

1413  7766776666667777  7777 

1313  66666566  67 


1304  66 
04  6 

1104  5 
CAP  04  6 


66776677 
7  7  6  5 

6  6  5  5 

7  6  7  6 


Vi  u  v  •  — 

1413  6655666655665555445555 
13  55665565566 

1213  65655566746 

1312  65  6565656554  6555 

1212  55756444  47 

1412  6  55555  56 

1413  7777777777777677  7777 

1313  76777667  77 

1413  55666566566 
1412  77  77  666666  6666 

12  66566  66 

1312  7  7  6  7  7  7  7 

1412  5  5  566  67 


1613  77666666666 


98 


321129012 

341129012  071079DE  2 

3011234591 

3111234591073079DP  1 

3211234591073079DP  1 

341123459 

3011257903071079PM  2 

311125790 

3211357903071279PM  1 

3411357903071279SD  1 

3011407141070979  022 

311140714 

3211407141071079YP  2 

341140714 

3011514011071679  042 

311151401  071879  042 

3211514011081879XB  2 

341151401 

3011543761071779  072 

3111543761071779  072 

3211543761071779  072 

341154376 

3011709503071379  091 

3111709503070979  091 

321170950 

3411709503070979  091 

3011751753070679  061 

311175175 
321175175 

3411751753072679  061 

3011929741070579PM 
3111929741070579PM  2 

3211929741070579PM  2 

3411929741070579PM  2 

3011942791070579  012 

311194279 

321194279  070579  012 
341194279 
3012027851 
311202785 

3212027851070979  012 

3412027851071279  2 

3012222321 

311222232  070979  032 

3212222321071379  032 

341222232 

XI 2356001 

311235600 

321235600 

341235600 


1512 

12  7  76776  77 

1212  7  77  76  77 

1405  5  4445 

0X5  2  2  2  3  5  4  4 

1305  11  1111 

1414  7666777766767675757676 

MAJ  14  77776777677 

1514  5666555555665566  65555 
14  77677677776 

1314  6  24451141 

1313  77  57767555565  6  6677 

13 

13  66664466666 

CAP  04  77777777 

04  77777767 

1104  6  7  7  7  6  7 

1304  65  76657665 


CAP  03  5  5  5  4  4 

6666766676557666666666 
14  65666666566 

1414  65667666666 
MAJ  14  65767665  66 

1414  6666666666666655556655 
65665666  66 

1314  76676777666 


1213  5  655571  66 
1413  5  55434  55 

12  75665666756 

1313  65765566666 
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Host  Frequent  (‘ummcnis,  1979  Managers  Lunj*  hurin  Kol  iuw-*i|i  O»i<?:,tionnaire 


appendix  j 


1979  FOLLOW-UP  QUESTIONNAIRE  FOR 
EMPLOYEES  RATED  ON  MPAS  FORMS, 
STUDY  TWENTY-SIX 


Lon*  Form  of  Follow-up  Questionnaire  on  Merit.  Promotion  Appraisal 

KKI- 1  •:  PK-WilTliS.  APPKAJSAi.  SYSTEM 

A J-/077 

FOLLOW-UP  QUESTIONNAIRE  Kill  EMPLOYEES  RATED  ON  ADD  FOKhS  387,  388,  and  389 

THIS  QUESTIONNAIRE  IS  VOUINTAKY  AND  ANONYMOUS 


As  part  of  the  implsnentat ior.  process  we  would  like  to  get  your  reaction  to  the 
new  Merit  Appraisal  form.  This  questionnaire  should  be  comp  It- red  only  by  those 
employees  appraiscJ  under  one  of  the  new  forms  ASD  387  (Fora  A),  388  (Fora  B), 
or  389  ( Form  C ) . 


Your  input  will  play  an  important  part.  :n  determining  if  changes  need  to  be  aade 
so  that  we  can  have  an  improved  Oorit  appraisal  system.  We  would  appreciate 
getting  your  frank  responses  to  the  questions,  as  well  as  any  additional  comments 
you  might  have. 

The  questionnaire  should  be  filled  out  anonymously.  Please  answer  the  questions 
as  hones'iy  as  possible,  if  you  have  difficulty  answering  a  question  with  the 
alternatives  given,  choose  the  one  that  comes  closest  to  your  own  opinion.  Jf 
you  do  n<  t  know  the  answer  to  a  question,  or  if  it  is  inapplicable,  please  leave 
it  blank.  However,  please  try  to  answer  all  items. 


Answer  e;.ch  question  in  the  space  provided  to  it's  right.  The  iv.ur.hers  in  paren¬ 
theses  refer  to  Cetunn  numbers  on  IBM  cards  for  the  use  of  the  key punchers. 


The  completed  questionnaires 

Kliould.br  mailed  to  Dr 

Steven  Norton,  ASD/DFCII 

within  five  days  of  receipt. 

Thank  you  very  much  tv 

If*  W*«lM  *  *T. . 

STUDY  NO 

(1) 

CAHI)  NO 

— 

(3/ 

record  NO 

(4) 

1.  Today's  date: 

MONTH 

(l.euve  Blank) 

(8) 

DAY 

(10) 

YEAk 

(12) 

— 

(14) 

2.  What  is  your  organizational  symbol? 

(e.g. _ A_  F_  A_  L_; _ E_  N_  2_: 

iLLLiii J 


3.  JKhat  is  you r  G,S.  Grade?  (e.g.  09)Median»  11  _  _  (22) 


4.  On  which  form  were  you  rated? 

B  »  Form  8;  C  »  Form  C) 

15*  33* 

5.  How  long  have  you  worked  at  AlUiV 
(Hound  off  U>  nearest  year;  lens  than  ttis 


(A  ■  Form  A;  *9* 


YKAKSi 


moiithr.  •  00  year::) 


M'th.ni  li 


(?*.•) 


113 


(i.  Mom  Inn/;  Iimvc  yen  S'Tvcil  in  your  preiu.-nl.  |x>::iLion 

(fit  i  Ik-  current  or  lower  jir.irti.-r. )  ?  Hcdi-m-  4  years;  VKAIC:  __  _  (?*/) 

7.  for  how  many  i  wined:  utc*  subord  i  n*i  ii*»  you  have 
a  direct  forma]  supervisory  responsibility? 

(If  none,  enter  "00" )  82S*  none  _  (29) 

8.  Did  your  supervisor  hove  a  feedback  interview 

with  you  on  the  merit  promotion  appraisal  ratings?  _  (31) 

(l«Yes;  2«No)  85**  Yon 

9.  If  yes,  about  how  many  minutes  did  the  interview 

last?  (e.g.  30  minutes  *  030)  Median*  lb  minutes  (32) 

10.  During  theoiinterview,  did  your  supervisor  discuss 
each  apprcisal  factor  and  explain  his  or  her  rating  of 

you?  (1  •  Yes;  2  *  No)  67%*  Yus  _  (35) 

11.  How  well  did  your  supervisor  handle  the  feedback 

interview':  (36) 


1  *  Viry  poorly  7*  4  «  Well  25* 

2  *  P.iorly  8-  5  =  Very  well  1TC 

3  »  Adequately  41* 


12.  How  much  information  do  you  have -regarding: 


1  *  No  information 

2  »  Some  information 

3  •  Fair  amounp,  of  information 

4  m  A  great  deal  of  information 


1 

2 

3 

4 

a. 

Where  the  completed  appraisal 

forma  go 

_ uiTIj 

30) 

30i 

— 

be 

How  the  appraisal  information 

is  stored 

_ 24 

41 

24 

10 

c. 

What  appraisal  information  is 

stored 

43 

35 

15 

5 

d. 

How  long  appraisal  information  is  stored 

47 

32 

14 

6 

e.  How- the  appraisal  information 
retrieved  by  a  manager 

can  be 

56 

28 

10 

5 

f. 

merit 

How  the  appraisal  information 
promotion 

la  used  in 

61 

24 

10 

4 

g.  What  is  involved  in  the  total 
promotion  process 

merit 

33 

42 

15 

8 

(38) 

(39) 

(40) 

(41) 

(42) 

(43) 

(44) 


2 
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13.  How  much  more  Information  would  you  like  to  have 
on  the  merit  promotion  process? 

12%  1  >  No  more  3  *  More  28% 


10%  2  * 

Somewhat  more  d  -  A  great  ue.il  more  4."i 

— 

(46) 

14.  Which  two  factors  did  you  find  most  difficult  to 
understand,  ir.  terms  of  why  you  received  a  given 
rating?  (Mark  a  "1"  after  each  of  the  two  most 
difficult  factors  to  understand.  Leave  the  others 
blank). 

a. 

Self  Management  8.5%  * 

(46) 

b. 

Work  administration  7.6% 

(47) 

c. 

Problem  Analysis  6.5% 

_ 

(48) 

d. 

Decii. 'an  Making  5.6% 

- . 

(49) 

e. 

Personal  Assistance  7.3* 

.  — * 

(SO) 

f. 

Info-nation  Processing  8-5* 

— 

(SI) 

«• 

Speaking  Ability  4.9% 

- — 

(52) 

h. 

Writing  Ability  4.9% 

— 

(  Ch>< 

1. 

Communication  Ability  5.3% 

_ 

(54) 

J- 

Working  Helation-sliips  •>.!•% 

_ 

(55) 

k. 

Work  Leadership  6.0% 

— 

(56) 

1. 

Personnel  Management  A  EEO  Commitment  13.2* 

w— 

(57) 

a. 

Performance  Under  Pressure  9-4* 

(68) 

n. 

Work  Output  6.7% 

— . 

(69) 

15.  Which  of  tbe  two  factors  did  you  find  moat  meaning¬ 
ful  in  terms  of  providing  you  with  greater  self-insight? 
(Mark  a  “1“  after  eacn  of  the  two  most  meaningful 
factors.  Leave  the  others  blank). 

• 

a. 

Self  Management  18.7%  • 

— - 

(60) 

b. 

Work  Administration  6,°* 

a. 

(61) 

c. 

Problem  Analysis  8-°* 

(62) 

d. 

Decision  Making  13.9* 

— 

(63) 

3 

*  %  of  number  of  timus  .-my  iuclor  ■ 

•MU  MU 

Mill.  • 

115 


Qucn u on  15  Con't 


e.  Personal  Assistance  l-7* 

f.  Information  Processing  ?.3 % 

g.  Speaking  Ability  3.7% 

h.  Writing  Ability  3.7% 

i.  Communication  Ability  5.0% 

j.  Working  Relationships  9,8% 

k.  Woik  Leadership  3.5% 

l.  Personnel  Management  t  EEO  Commitment  -29% 

a.  Pei  Airmanee  Under  Pressure  10.0% 

n.  Work  Output  10.9% 


STUDY  NO 
CARD  NO 
RECORD  NO 


16.  As  a  result  of  the  new  merit  appraisal  system 
did  your  supervisor: 

1  m  Much  less  than  before 

2  *  Leas  than  before 

3  •  The  same  as  before 

4  m  More  than  before 

5  ■  Much  more  than  before 

a.  Have  a  better  understanding  of  your 
potential  for  higher  level  work? 

b.  Have  a  better  understanding  jf  how 
you  perform  your  job? 

c.  Have  a  better  understanding  of  you 
as  an  individual? 

d.  Better  indicate  recognition  of  your 
good  work? 


(Leave  blank) 


? 

3 

4 

3r' 

7b% 

16% 

3 

78 

13 

4 

09 

19 

», 

Ti 

e 


Hotter  utilize  your  particular  okilluV 


f.  Take  a  greater  personal  interest  in  you 
and  your  future? 

g.  Make  a  greater  effort  to  help  you  develop 
yoursel f? 

h.  Refer  to  your  Individual  Development 
Plan  (AfSC  Form  153)? 


1 

X 

3 

4 

5 

_ 3>. 

10% 

1*  (12) 

4 

4 

75 

13 

3  (13) 

A 

3 

74 

14 

3  (14) 

5 

6 

72 

14 

2  (15) 

18.  Do  you  have  a  better  picture  of  what  your  supervisee 
expects  of  y  u  in  terns  of  demonstrating  potential  for 

higher  level  work?  (l*Yes;  2»No)  45%*  Yes  __  (16) 

IS'.  Do  you  have  a  better  picture  of  your  overall  stand¬ 
ing  with  you;  supervisor?  (l*Yes;  2-tlo)  55%*  Yes  _  (17) 


20.  Did  you  have  a  greater  opportunity  than  in  the  past 
to  present  your  side  of  the  story  during  the  discussion?  . 

(l.Yes;  2*No)  41%*  Yes  _  (18) 

20.  The  ASD  Commander  sent  out  a  letter  to  all  of  the 
ASD  serviced  organizations.  Thin  letter  described 
his  concern  with  i  he  muht*  e.. -f  the 
promotability  appraisals  to  he  done  this  year.  It 
also  described  his  charge  to  the  two  iettcr  Commanders/ 

Chiefs/Directors  to  be  responsible  for  quality  control 
over  the  rating  process. 


Which  of  the  following  best  describes  what  you 
know  of  this  letter  when  you  discussed  your  appraisal 
with  your  supervisor? 

1  »  I  hadn't  heard  of  it  34* 

2*1  had  not  seen  a  copy  of  the  letter  26% 

3*1  had  seen  a  copy  of  the  letter  39% 


21.  Do  you  think  that  the  quality  control  procedures 
this  year  are  an  improvement  over  last  year?  (l*Yea;  17% 
2«No;  3*Don’t  know) 

27%  56% 

22.  Do  you  think  ASD  Management  is  serious  about  control 
of  the  quality  and  fairness  of  the  numerical  appraisals? 
(l*Yes;  2*No;  3»Don*t  know) 

27%  35%  36% _ 

23.  How  much  impact  do  you  think  numerical  ratings 
given  on  the  appraisal  form  hove  on  your  personal 
progress? 


33%  1  *  Very  1  Itl  l-' 
10%  2  *  Utile 


>1 


«'■ 

A  ;*.hMl  iliM I  * 


(19) 


(20) 

(21) 
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24.  How  much  opportunity  does  your  job  offer  for 
displaying  your  ability  in  respect  to  the  appraisal 
factors? 


(22) 


14%  1  a  Very  little 


9%  2  a  Little 


3  >  Some  42% 


4  *  k  great  deal  34% 


25.  In  your  experience,  do  you  think  the  merit  appraisal 
ratings  were  performed  with  conscientious  use  of  the 
factor  definitions  and  rating  steps  in  the  new  form? 


10%  i  m  Definitely  not 
20%  2  •  Pro!  ably  not 


3  m  Probably  yes  56% 

4  »  Definitely  yes  13% 


26.  Based  or  the  wording  of  the  scale  steps  do  you  think 
the  ratings  >ou  received  on  the  new  merit  appraisal  fora 
should  have  been: 

.4%  i  .  Much  lower  than  they  were  4  «  Higher  33% 


5%  2  »  Low  ir 

55%  3  a  The  same  as  they  were 


5 2  3 *  Much  higher  than 
they  were  6% 


*/.  compared  to  wnat  you  tninn  otner  peoples'  ratings 
may  have  been,  do  you  think  the  ratings  you  received  on 
the  new  merit  appro isal  form  should  have  been: 

OX  la  Much  lower  than  they  were  4  *  Higher  44% 


.89X2  «  Lower 

47%  J  a  The  same  as  they  were 


5  *  Much  higher  than 
they  were  8% 


28.  Was  the  rating  your  immediate  supervisor  wanted  to 
give  you  changed  by  the  reviewing  supervisor  before 
you  signed  the  rating?  (l=Yes;  2«No ;  3«Don't  know) 

_ _ 10%  40%  49% 

29,  In  your  opinion,  have  your  promotion  ratings  been 
affected  by  the  visibility  of  your  project  to  higher 
managesient? 

1  m  The  project's  visibility  has  helped  support 

high  ratings  22* 

2  »  The  project's  degree  of  visibility  has  had 

no  effect  on  my  ratings  50% 

3  -  The  project's  lack  of  visibility  has  made  it 

more  difficult  for  mr  l.n  ivrnivc  hlrJi  rtilliMin. 
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30.  The  introduction  of  the  new  Appraisal  Forms  involved  much 
more  intensive  training  than  has  been  provided  in  the  past. 

What  impact  has  the  new  form  had  cn  your  confidence  in  the 
Merit  Promotion  process  as  a  source  of  viable  promotion 

opportunities?  _  (28) 

1  -  The  new  form  has  decreased  my  confidence  11% 

2  =  The  new  form  has  had  no  effect  on  my  confidence  70% 

3  »  The  new  form  i.a-i  increased  my  confidence  13% 

31.  How  well  did  the  form  on  which  you  were  appraised  fit 
your  own  job?  That  is.  were  the  factors  on  which  you  were 

appraised  relevant  to  your  job?  __  (29) 

1  »  Mi  eh  worse  than  it  was  under  the  old  system  1% 

2  »  Stirewhat  worse  than  it  was  under  the  old  system 5% 

3  «  No  difference  42%  .. 

-  4  a  Somewhat  better  4  * 

5  «  Much  better  8% 

32.  Taken  on  the  whole,  what  is  your  reaction  to  the  new 

merit  proiotion  appraisal  form?  .  _  (30) 

1  «  I  dislike  the  new  form  ** 

2  •  I  have  a  somewhat  negative  reaction  to  the  new  form  11% 

3  a  1  feel  neutral  about  the  new  form  52% 

4  a  I  ,iave  r  «'"-*•*  - ,e«.uu n  to  the  new  form  28% 

5  a  I  am  very  positively  impressed  by  the  new  form  4% 

33.  Do  you  think  the  ratings  given  by  supervisors  in  your 
two-letter  organization  can  be  fairly  compared  to  the 
ratings  given  by  supervisors  in  other  organizations? 

UaYess  2bNo)  47%-  Yes  _  (31) 


(32) 


(33) 

36.  Do  you  know  what  the  performance  standards  are  for 
your  Job?  (1  a  Yea;  2  a  No;  3  »  Don’t  know)  _  (3*) 

52%  32%  15% 


i  34.  Co  you  think  that  mu:;t  managers  performed  the  merit 
I  appraisal  ratings  with  conscientious  use  of  factor 
;  definitions  and  rating  steps  in  the  new  form? 

1  a  Definitely  not  10%  4  a  Definitely  yas  5* 

2  -  Probably  not  28%  5  a  Don’t  know  10% 

3  »  Probably  yes  *?* 


35.  Do  you  think  it  would  be  desirable  to  have  written 
performance  standards  for  your  job?  (1  -  Yeaj  2  ■  No; 
3  ■  Don’t  know)  24%  53*  23* 
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37.  Are  the  performance  ttancard^  for  your  job 

written?  (1-Yes;  2-No)  32-».  Yea 

38.  Did  you  participate  in  creating  cr  Retting  the 

performance  standards  for  your  job?  (uyes;  2-No; 
3-Don ' t  know)  14x  77X 

9% 

39.  Have  you  received  feedback  on  how  your  performance 
compares  to  the  performance  standards  for  your  iob? 
(1-Yes;  2-No)  31%=  Yes 


40.  Do  you  know  of  any  specific  cases  in  which  a 
rating  may  have  been  influenced  by  one  of  the 
following  factors  outside  of  the  employee's  potential 
for  promotion?  (1-Yos;  2-No) 

a.  Opportunity  to  demonstrate  potential  limited 
by  job  itself  3V'—  Yes 

e 

b.  Sut  ervisor  could  not  observe  employee  22%»  Yes 

c.  Supervisor  lacked  technical  knowledge  to 
judge  employee  26%«  Yes 

d.  Supervisor  was  influenced  by  employee's 
race,  religion,  sex,  or  age  Aj%-  Yes 

I - - - - 

«.  nig.ier  ratings  were  given  to  other  employees 
because  the  employee  was  not  likely  to  be 
considered  for  competitive  promotion  (for 
example,  because  of  technical  specialty, 
time  in  grade,  etc.)  43%-  Yes 


f.  Other  (Pleaae  describe) 


(35) 

(36) 


(37) 


(38) 

(39) 

(40) 

(41) 


(42) 

(43) 


41.  Do  you  have  any  comments  on  the  implementation  of  the 
Appraisal  Form  in  your  organization,  or  comments  about  the 
Merit  Promotion  System  itself?  Specifically,  what  factors 
will  determine  the  degree  of  success  of  this  year's  implemen¬ 
tation  of  the  new  appraisal  form?  What  changes,  if  any,  do 
you  think  should  be  made?  (Please  use  a  separate  sheet  if 
necessary.  Return  questionnaire  to  ASD/DPCH  (Norton)). 
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Most  Frequent  Comments,  1979  Long  Form,  f.mployees  Follow-up  Questionnaire 

1077  employees  total 
132  comments  total 


Comment 


%  of  all  cowmen t» 


s. 

Favorable  toward  new  form 

5.30 

19. 

Merit  promotion  is  a  justification  system  for  management 
playing  favorites 

9.09 

20. 

Promotions  are  practically  non-existent  for 
people 

professional-level 

5.30 

24. 

Appraisals  have  little  or  no  effect  on  promotions;  new  system 
no  better  than  old  because  politics  destroy  equity 

11.36 

26. 

Supervisors  need  to  learn  how  to  rate 

5.30 

29. 

Covert  quota  system  is  in  use  and  should  be 

eliminated 

9.09 

33. 

Miscellaneous  comments  on  administration  of 
negative) 

ratings  (mostly 

31.82 

Comments  with  less  than  5%  endorsement 

22.74 

Total 

100.00 
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Fig.  K.l  Aeronautical  Systems  Division — Professional 

1978  Ratings 


123 


1HKUW28'  Oh  B.UC10OZZUJ 


Fig.  K.2  Aeronautical  Systems  Division — Professional 

1979  Ratings 
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Fig.  K.3  Aeronautical  Systems  Division — Professional 

1980  Ratings 
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Fig.  K.4  Aeronautical  Systems  Division — Technical 
1978  Ratings 
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Fig.  K.9  Aeronautical  Systems  Division— Clerical 
1980  Ratings 
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Fig.  K.ll  Aeronautical  Systems  Division — All 
1979  Ratings 
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2713 


Fig.  K.12  Aeronautical  Systems  Division — All 
1980  Ratings 
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Fig.  K.1”  4950th  Test  Wing 
1978  Ratings 
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dniivh  tvs: 


Fig.  K.15  4950th  Test  Wing 
1980  Ratings 


Fig.  K.16  Foreign  Technology  Division  (FID) 
1978  Ratings 


Fig.  K.17  Foreign  Technology  Division  (FFD) 
1979  Ratings 
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(X.  W  K  O  W  2  E->  Oh  fcWBWOZZMJ 

Fig.  K.19  Air  Force  Wright  Aeronautical  Laboratory  (AFWAL) 

1978  Ratings 
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Fig.  K.20  Air  Force  Wright  Aeronautical  Laboratory  (AFWAL) 

1979  Ratings 
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APPRAISAL  RATING 


M 

Ou 

Fig.  K.22 
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W  OS  O  W  2  H  Ob  CU  W  K  W  O  2  Z  W  J 

Air  Force  Aerospace  Medical  Research  Laboratory 
1978  Ratings 
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APPRAISAL  RATING 
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Fig.  K.24  Air  Force  Aerospace  Medical  Research  Laboratory 

1980  Ratings 
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Fig.  K.25  Air  Force  Hunan  Resources  Laboratory 
1978  Ratings 
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Fig.  K.26  Air  Force  Human  Resources  Laboratory 
1960  Ratings 
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APPRAISAL  RATING 
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Fig.  K.31  All  Serviced  Activities 
1980  Ratings 
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APPRAISAL  RATING 


APPENDIX  L 

MULTITRAIT-MULTIMETHDD  MATRIX 
FOR  FORM  A-ALL 
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Trait 

Designation 


Trait  Description 


£ 

D* 

I 


Self  Management 
WorSc  AcfrrcLnistration 
Problem  Analysis 
Decision  Making 
Speaking  Ability 
Writing  Ability 
Working  Relationships 
Work  Leadership 
Personnel  Management 
Performance  Under  Pressure 
Work  Output 
Overall  Rating 


Tb  construct  Table  L.l,  ascending  pages  should  be  laid  side 
by  side  from  left  to  right. 


Construction  of  Table  L.l 


Table  L.l 

i 

i 

i 

1 

Table  L.l-Cont. 

• 

Table  L.l-Conte 

Table  L.l-Cont. 

i 

l 

i _ _ _ 

. 

i 

i 

/ 

I 

/ 

< 

Page  156 

Page  157 

_ 

Page  156 

Page  159 

Note:  Due  to  the  varying  number  of  cases  used  in  computing  the  corre¬ 
lation  coefficients,  the  number  of  higher  significant  values  in 
Tables  4.3  and  4.4  mbs  summarized  from  the  significance  levels 
of  the  correlation  coefficients  in  Table  L.l.  The  technique  of 
comparing  the  magnitudes  of  the  correlation  coefficients  can 
only  be  used  vhen  the  number  of  cases  is  constant  for  all 
calculations. 
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TABLE  L.l 

MULTITRAIT-MULTIMETHOD  MATRIX 


Actual  Ratings 


\ 

B1 

C1 

D1 

E1 

F1 

G1 

«1 

X1 

J1 

«1 

4 

A, 

(  ) 

(  ) 

B1 

.56 

C1 

.68 

.60 

(  ) 

A 

.66 

.68 

.86 

(  ) 

C 

§ 

.24 

.51 

.46 

.48 

(  ) 

T 

.44 

.61 

.65 

.70 

.69 

(  ) 

U 

.12 

.78 

.36 

.37 

.51 

.33 

(  ) 

A 

£ 

.59 

.70 

.48 

.37 

.37 

.31 

.64 

(  ) 

L 

It 

.60 

.65 

.39 

.34 

.42 

.36 

.58 

.74 

(  ) 

.55 

.63 

.55 

.52 

.63 

.50 

.48 

.72 

.59 

(  ) 

kJ 

.50 

.32 

.48 

.39 

.39 

.38 

.26 

.64 

.65 

.63 

(  ) 

H 

.70 

.84 

.81 

.80 

.73 

.78 

.61 

.79 

.77 

.82 

.66 

(  ) 

m 

,*24 

-.24 

-.38 

.27 

-.18 

.08 

.60 

.70 

.03 

.21 

.09 

B§ 

.52 

r7cl 

.  -03. 

-.16 

.28 

.36 

.30 

.63 

.56 

.27 

.44 

.49 

C2 

.30 

.13 

-.21 

.29 

-.07 

-.02 

.49 

.49 

.02 

.09 

.05 

°2 

.51 

.60 

.12 

Eisl 

A 

.27 

.22 

.63 

.70 

.30 

.07 

.40 

S 

E? 

-.16 

.32 

-.23 

-.21 

.36 

.27 

.35 

.42 

.23 

.00 

.27 

E 

F? 

.21 

.66 

.20 

.15 

.55 

GU 

.30 

.31 

.38 

.15 

.06 

.38 

L 

-.12 

.14 

-.08 

.06 

.45 

.42 

mu 

.43 

.26 

-.05 

.00 

.22 

F 

«2 

-.07 

.31 

-.33 

-.17 

.53 

.12 

,36 

1.551 

.68 

.03 

.03 

.29 

i; 

.44 

.64 

.00 

.24 

.17 

.44 

.17 

.32 

mu 

.24 

.37 

.48 

.53 
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